01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.9 / 30
03Economics
19.1 / 20
04Discoverability
13.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
6 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
handoff_stats |
no_return_description |
Get storage statistics and current storage limits. Returns usage data for the handoff store (e.g., number of stored handoffs, conversation byte sizes, message counts, and remaining capacity against the default 1 MiB per-handoff limit), useful for monitoring storage usage. |
handoff_restart |
no_return_description |
Restart the shared HTTP server and return a result confirming success or reporting failure. Use when the server is in an unhealthy state. Note: all stored handoffs will be lost, since data is kept in-memory. |
handoff_merge |
name_restates_behavior no_return_description |
Combine multiple handoffs into a single unified handoff by merging their conversations and metadata, in chronological or original array order. Use this to consolidate related handoffs (e.g., parallel conversations about the same project) into one record before handing off. Returns the merged handoff (with its new key and title); source handoffs can optionally be deleted after merging. |
handoff_add_comment |
name_restates_behavior no_return_description |
Attach a comment or annotation to an existing handoff, optionally with an author name (defaults to 'anonymous'). Comments are included whenever the handoff is loaded, so use this to record notes or feedback alongside the conversation. Returns a confirmation that the comment was added. |
handoff_delete_comment |
name_restates_behavior no_return_description |
Delete a specific comment from a handoff using its comment ID. Returns a confirmation that the comment was removed, so it will no longer be included when the handoff is loaded. |
handoff_append |
no_return_description |
Append a conversation chunk to an existing handoff. Use this to upload long conversations in multiple pieces, e.g., when the conversation exceeds ~100KB and handoff_save fails with an XML parse / tool-input error. Call handoff_save first, then call this repeatedly; chunks are concatenated as-is, so include any newlines or separators. The cumulative conversation size must stay within the storage limit (default 1 MiB). Returns a confirmation that the chunk was appended. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
handoff_list |
handoff_load |
medium | Both retrieve handoff data; a task like 'show me my handoffs' or 'view the handoff' is ambiguous between listing summaries (list) and returning full content (load). |
handoff_clear |
handoff_restart |
medium | Both destroy stored handoffs; a task like 'reset/wipe all handoffs' could select restart (which explicitly loses all in-memory data) instead of clear, or vice versa. |
handoff_add_comment |
handoff_delete_comment |
low | Both operate on comments with shared tokens; a task like 'update/change the comment on handoff X' is ambiguous because neither edits, and 'remove a note' could be mistaken for add_comment by an agent skimming descriptions. |
handoff_clear |
handoff_merge |
low | A task like 'clean up / consolidate duplicate handoffs' could map to merge (combining them) or clear (removing them), since both are cleanup-style operations on existing handoffs. |
handoff_save |
handoff_add_comment |
low | A task phrased as 'save a note/annotation about this project' could select handoff_save (creating a stored handoff) instead of handoff_add_comment (annotating an existing one), since 'save' and 'note' overlap semantically. |
Compare the field