01Safety
0.0 / 30
What changed in the harness
Selection accuracy 92→90, token cost up 2%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.6 / 30
03Economics
20.0 / 20
04Discoverability
10.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
message_human_operator |
no_return_description |
Send a free-form message to the human operator — questions, scoping, custom or recurring projects, anything that is not yet a ready-made task. The call returns an acknowledgment that the message was sent; the operator's substantive reply is delivered by email to reply_to (REQUIRED, a real, reachable MX-checked mailbox), which is the only way the operator can answer, so it is not returned from this call. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_human_services |
message_human_operator |
low | A task phrased as 'ask the operator what services/response times they have' could route to message_human_operator, but get_human_services explicitly returns the catalog, profile, and response times, so agents reading descriptions normally pick the manifest fetch. |
get_human_services |
submit_human_task |
low | Fetching a catalog versus submitting a hire request are distinct actions with different schemas (no inputs vs task_type/description); a request to hire the human unambiguously maps to submit_human_task, so confusion is unlikely. |
submit_human_task |
check_task_status |
low | Submitting a task (task_type/description) and polling status (task_id) have disjoint required inputs and verbs; no natural task plausibly swaps create for lookup. |
submit_human_task |
message_human_operator |
medium | Both target the human operator. Instructions like 'get the operator to verify this address' or 'ask the human to review my product' could fit submit_human_task (human_judgment_and_feedback / custom_human_in_the_loop) or message_human_operator, which explicitly covers scoping and 'anything that is not yet a ready-made task' — a genuine overlap. |
Compare the field