01Safety
0.0 / 30
What changed in the harness
Selection accuracy 83→69, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.1 / 30
03Economics
20.0 / 20
04Discoverability
14.7 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
preview_markdown |
no_return_description |
Preview markdown content in MarkView. Writes content to a temp file and opens it in the native macOS MarkView previewer with live reload. Returns the temp file path that was created and opened. |
open_file |
no_return_description |
Open an existing markdown file in MarkView for live preview. Returns confirmation that the file was opened, including the path used. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
render_diff_file |
render_diff_raw |
high | Both render a git diff via diff2html with identical format options; a task like 'show me the diff for this repo' is ambiguous about whether the agent should run render_diff_file(path) or first obtain diff text and pass it to render_diff_raw(diff), especially if the agent already has diff text on hand. |
lint_file |
lint_content |
high | Identical linting functionality differing only in whether input is a file path or raw content; a task like 'lint this markdown' without specifying source (e.g., agent has both a path and the content in context) could plausibly trigger either tool. |
preview_markdown |
open_file |
medium | Both open markdown in the MarkView previewer; a task like 'preview this markdown file' is ambiguous between opening the existing file directly (open_file) versus reading its content and passing it to preview_markdown, especially since preview_markdown writes to a temp file. |
open_file |
lint_file |
low | Both take a file path and markdown context, but their actions (preview vs lint) are distinct enough that a natural request ('check this markdown file') rarely conflates opening a live preview with running diagnostics. |
preview_markdown |
lint_content |
low | Both accept raw markdown content, but 'preview' vs 'lint' intents are usually clear from the task phrasing, making confusion less likely than the content/file-based lint pair. |
Compare the field