01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 5%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
29.0 / 30
03Economics
20.0 / 20
04Discoverability
11.0 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
fixgraph_get_fixes |
no_return_description |
Fetches a FixGraph issue by its ID or slug (from a fixgraph_search result) and returns all verified fixes associated with it, including their details and how to apply them. |
fixgraph_submit_fix |
name_restates_behavior no_return_description |
Creates a new fix attached to an existing FixGraph issue, supplying its title, root cause, ordered fix steps, validation method, and risk level. Requires the FIXGRAPH_API_KEY environment variable. Returns the outcome of the submission so callers can confirm the fix was recorded. |
Selection evidence
Confusable tool pairs.
1 pair where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
fixgraph_search |
fixgraph_get_fixes |
medium | Both tools present as read-only 'fix lookup' operations and both descriptions prominently mention 'fixes'. A user task like 'get the fixes for issue abc-123' could route to fixgraph_search because its description says 'Search FixGraph for engineering fixes' and it is the more general look-up tool, even though the correct choice (fixed issue_id input) is fixgraph_get_fixes. Since fixgraph_search never accepts an issue_id and only returns issues (not fixes), an agent paraphrasing 'get fixes' may pick the wrong one unless it notices the issue_id semantics. |
Compare the field