01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 3%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
29.5 / 30
03Economics
20.0 / 20
04Discoverability
20.0 / 20
Highest-impact fix
Estimated gain +1 pointRewrite 1 underspecified tool description
State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
generate_soul_template |
no_return_description |
Generate a starter SOUL.md file for an agent, returning the generated file content as a string. The starter is seeded from the agent's name and optional keywords describing its domain or expertise. |
Selection evidence
Confusable tool pairs.
1 pair where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
validate_soul_file |
score_soul_file |
medium | Both tools take the identical 'path' input targeting a .soul.md file and both perform an 'evaluation-style' operation on it, so a natural task like 'check my SOUL.md' or 'evaluate my soul file' could route to either, especially since selectors must infer whether the user means schema validation or completeness scoring. |
Compare the field