01Safety
0.0 / 30
What changed in the harness
Selection accuracy 90→93, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.3 / 30
03Economics
19.4 / 20
04Discoverability
12.0 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
update_status |
name_restates_behavior |
Advance a change request through its lifecycle (new, accepted, building, preview, approved, shipped, or rejected) so downstream tools and the requester see the current stage. Takes the ticket id and the new status string, and returns the request code with the status now set. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
set_preview_url |
publish_preview |
high | Both mark a change request ready for requester review and take the same 'id' parameter; a task like 'mark RB-3 ready for review' doesn't indicate whether the preview is externally hosted (needs a URL) or already live from the worktree, so an agent could easily pick the wrong one or fail to realize set_preview_url requires a URL argument. |
get_change_request |
ship_change |
low | Both operate on a single change request by id, but their purposes (fetch implementation detail vs. merge and deploy) are clearly distinct in description, making confusion unlikely except in very vague prompts like 'handle RB-3'. |
list_change_requests |
ship_change |
low | list_change_requests is a read-only listing tool while ship_change performs a merge/deploy action on one id; the differing signatures (no id vs required id) and distinct verbs make accidental selection unlikely. |
Compare the field