01Safety
30.0 / 30
What changed in the harness
Selection accuracy 95→98, token cost up 7%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.1 / 30
03Economics
19.8 / 20
04Discoverability
17.5 / 20
Highest-impact fix
Estimated gain +1 pointRewrite 5 underspecified tool descriptions
State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.
Description evidence
Defects and rewrites.
5 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
obsidian_simple_search |
no_return_description |
Returns a list of up to max_results matching documents in the vault, with each result showing the matching text plus context_length characters of surrounding content. The search is case-insensitive and covers all file content. Use offset together with max_results to paginate through results, and rely on this tool for plain keyword lookups rather than structured metadata queries. |
obsidian_get_file_content |
no_return_description |
Returns the complete raw text content of the note located at the given filepath relative to the vault root (e.g., 'notes/my important note.md'). Use this after locating a file via search or file listing when you need to read its full body. |
obsidian_list_files |
no_return_description |
Returns the files and folders present in the given vault-relative directory (or the vault root when directory_path is omitted), including each item's name, size, and created/modified timestamps. Results can be sorted by name, modified, created, or size in either order, and max_items with offset paginate through large directories. |
obsidian_fuzzy_search |
no_return_description |
Returns a list of up to max_results notes that approximately match the query, tolerating typos and imprecise spelling or phrasing. Works without the obsidian api, so it is a good fallback for finding notes when you are unsure of the exact wording. |
obsidian_graph_search |
no_return_description |
Returns the network of links and connections among notes in the vault: starting from start_file when provided, it analyzes relationships up to max_depth hops, identifies orphaned notes (notes with no links) when include_orphans is true, and reports the resulting link graph. Works without the obsidian api. |
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
obsidian_dataview_search |
obsidian_fuzzy_search |
medium | Both accept a free-text 'query' and can locate notes by name or content, so a vague request like 'find notes related to X' doesn't reveal whether structured DQL or approximate keyword matching is intended. |
obsidian_dataview_search |
obsidian_graph_search |
high | Dataview explicitly handles backlinks and note relationships ('list from [[FILE]]') while graph search analyzes links and connections, so tasks like 'find what links to X' or 'show note relationships' map to both tools. |
obsidian_simple_search |
obsidian_dataview_search |
medium | Both take a 'query' and search vault content and titles; a plain 'search for X in my vault' request doesn't make clear whether a simple text match or a DQL query is wanted, and both descriptions mention 'search'. |
obsidian_simple_search |
obsidian_fuzzy_search |
medium | Both perform text matching over notes using a query string, so 'find notes mentioning [term]' is ambiguous between exact/simple matching and approximate/fuzzy matching, especially when the user has typos or uncertain phrasing. |
obsidian_get_file_content |
obsidian_list_files |
medium | A request like 'show me the contents of folder X' is genuinely ambiguous: 'contents' could mean listing the folder's entries (list_files) or retrieving the content of files (get_file_content). |
obsidian_dataview_search |
obsidian_list_files |
medium | 'List all notes in folder X' can be satisfied either by dataview's folder search ('list from "folder/"') or by list_files with a directory_path, so the agent could plausibly select the wrong one. |
Compare the field