01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 2%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
29.4 / 30
03Economics
20.0 / 20
04Discoverability
18.6 / 20
Highest-impact fix
Estimated gain +1 pointMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
file_contributors |
no_return_description |
Returns the full list of every author who has ever committed to a file, ranked by number of commits. For each author it provides their commit percentage of the file's history, current line ownership, and the active date range during which they committed to the file, enabling identification of the file's primary maintainers and when they were active. |
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
commit_story |
search_commits |
low | Definitions cleanly split a hash-keyed lookup vs a keyword search, so most tasks map unambiguously. Confusion is only plausible for a task like 'tell me the full story/details of the commit that fixed X', where the agent has no hash and might call commit_story directly instead of search_commits; genuine wrong-pick risk is low. |
file_history |
file_contributors |
medium | Both take the same file-path input and concern commits touching a file, so a task phrased such as 'who has committed to this file' or 'who has been working on this file' could plausibly route to the wrong one: history returns the recent commits, contributors returns the ranked authors. The line between 'commits on this file' and 'people on this file' is thin in natural language. |
Compare the field