01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 9%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.9 / 30
03Economics
20.0 / 20
04Discoverability
14.0 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
4 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
task_list |
name_restates_behavior no_return_description |
Query the shared 5dive task queue and return the matching tasks (id, title, status, priority, assignee). Open tasks are returned by default in priority order; set status to filter by lifecycle state (e.g. todo, in_progress, blocked, done), assignee to filter by owner name, or pass all=true to include done/closed tasks as well. |
agent_send |
name_restates_behavior no_return_description |
Deliver an inter-agent message to the named agent on the fleet; the recipient receives it in-session. Use this for cross-agent coordination and rely on the returned confirmation to verify the message was sent. Optionally set from to label the sender. |
agent_list |
name_restates_behavior no_return_description |
Return every agent currently registered on the box, each with its name, type, channels, model, and live state, so callers can see who is active and how to reach them. |
digest_get |
no_return_description |
Fetch the fleet's daily standup digest and return its contents covering activity, token burn, and health. The default window is 1d (daily); pass window=7d to retrieve the weekly digest instead. |
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
task_show |
task_list |
medium | Both read the same task entity and a user asking to 'show me the open tasks' or 'show the task queue' is likely to be matched to task_show purely on the name ('show'), even though task_list is correct and is the only tool supporting status/assignee filters. The descriptions disambiguate, but the surface phrasing overlap makes wrong selection plausible. |
task_list |
agent_list |
low | Both are broad 'list' operations sharing the 'list' token, so a vague request like 'list what's on the box' or 'give me the whole list' is ambiguous. However the tool descriptions clearly separate tasks-in-queue from agents-on-the-box, so most natural-language tasks name the entity explicitly and resolve correctly. |
Compare the field