01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 6%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.4 / 30
03Economics
20.0 / 20
04Discoverability
15.3 / 20
Highest-impact fix
Estimated gain +5 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
downloads |
no_return_description |
Get download statistics for an npm package. Returns the download count for the specified package over the given period (one of last-day, last-week, last-month, or last-year; defaults to last-month). |
compare_downloads |
name_restates_behavior no_return_description |
Compare download counts across multiple npm packages (2 to 10) to gauge relative popularity. Returns the download count for each package in the input list over the selected period (one of last-day, last-week, last-month, or last-year; defaults to last-month). |
vulnerabilities |
no_return_description |
Get vulnerability info for an npm package. Returns the known vulnerabilities and advisories for the specified package. Note: Full audit requires npm audit in project context. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
downloads |
download_trends |
medium | Both tools fetch download stats for a single npm package with identical schemas and period options; a task phrased as 'show me the download history/trend/sparkline for lodash' or 'are react's downloads growing' could select the wrong one since neither name nor description makes the distinction salient without reading details. |
compare_downloads |
download_trends |
medium | A task like 'compare the download trends across lodash and express' is ambiguous: the agent might call compare_downloads (which returns aggregate counts) or download_trends (which returns per-package daily data) because both involve comparing downloads and only one accepts multiple packages. |
downloads |
compare_downloads |
low | Tasks naming two or more packages clearly point to compare_downloads, and single-package queries clearly point to downloads; confusion is only plausible when the user says 'get the downloads for react and vue' without the word compare, though the schema strongly disambiguates. |
Compare the field