01Safety
0.0 / 30
What changed in the harness
Selection accuracy 95→93, token cost up 3%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.3 / 30
03Economics
20.0 / 20
04Discoverability
13.1 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
preview_markdown |
no_return_description |
Preview markdown content in MarkView. Writes content to a temp file and opens it in the native macOS MarkView previewer with live reload. Returns the path to the temp file created for the preview. |
open_file |
no_return_description |
Open an existing markdown file in MarkView for live preview. Returns a confirmation that the file was opened in the previewer. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
render_diff_file |
render_diff_raw |
high | Both render diffs in MarkView with diff2html. A task like 'render/show the diff in MarkView' is genuinely ambiguous about the input source: a repo path (render_diff_file runs git diff itself) versus a raw unified diff string the agent already has (render_diff_raw). If the user doesn't specify whether the diff should be computed from a repository or passed directly, an agent can pick the wrong one. |
lint_file |
lint_content |
medium | Both run the same 9-rule linter and differ only by input source (file path vs raw content). A natural task like 'lint this markdown' is ambiguous when the agent has content in context but a path also exists, or vice versa — the verb 'lint' alone doesn't disambiguate whether to pass the path or inline content. |
preview_markdown |
open_file |
medium | Both open markdown in the native MarkView previewer with live reload, and differ only by input form (inline content vs existing file path). A task such as 'preview this markdown' is ambiguous if the agent has both the file content and a path, or if the user expects a temp-file preview when a real file exists. |
render_diff_file |
get_changed_files |
medium | Both operate on a git repository given a path. A task like 'show me what changed in repo X' or 'show my changes' could plausibly be answered by listing changed files (get_changed_files) or by rendering the actual diff (render_diff_file), so an agent may select the wrong member based on how it interprets 'changes'. |
preview_markdown |
lint_content |
low | Both consume raw markdown content, and a vague task like 'check/review this markdown content' could mean render a preview or run lint diagnostics. However, 'lint' intent (errors/warnings) vs 'preview' intent (visual display) are described distinctly enough that real confusion is only marginal. |
Compare the field