01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 13%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.1 / 30
03Economics
20.0 / 20
04Discoverability
13.1 / 20
Highest-impact fix
Estimated gain +7 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
7 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
package_info |
no_return_description |
Fetch metadata for an npm package by name. Returns an object with fields like description, license, repository URL, and dependencies. |
downloads |
no_return_description |
Fetch npm download counts for a package over a given period (last-day, last-week, last-month, or last-year; defaults to last-month). Returns the total download count for that period. |
compare_downloads |
name_restates_behavior no_return_description |
Fetch npm download counts for 2-10 packages over the same period (last-day, last-week, last-month, or last-year; defaults to last-month) so their popularity can be compared side by side. Returns a mapping of each package name to its download count for that period. |
bundle_size |
no_return_description |
Look up the minified and gzipped bundle size of an npm package (optionally a specific version, e.g. 'lodash@4.17.21') via the Bundlephobia API. Returns the package's minified size and gzipped size in bytes. |
vulnerabilities |
no_return_description |
Look up known security vulnerabilities for an npm package by name. Returns a list of known vulnerabilities (e.g. severity, advisory details) for that package; note a full audit requires npm audit in project context. |
dependency_tree |
no_return_description |
Look up the direct dependencies of an npm package by name. Returns a list of the package's direct dependencies and their version ranges (not a full recursive tree). |
download_trends |
no_return_description |
Fetch a daily breakdown of npm download counts for a package over a given period (last-day, last-week, last-month, or last-year; defaults to last-month). Returns a list of daily download counts and a sparkline-style summary of the trend. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
downloads |
compare_downloads |
medium | A task like 'get download stats for express, react, and vue' could plausibly be routed to single-package 'downloads' called repeatedly instead of 'compare_downloads', since both fetch download counts and differ mainly by singular vs plural package input. |
downloads |
download_trends |
medium | A request like 'show me download activity for lodash over the last month' is ambiguous between raw stats (downloads) and a daily breakdown/sparkline (download_trends), since both operate on a single package and period with overlapping phrasing. |
compare_downloads |
download_trends |
low | Both involve download data over a period, but one is inherently multi-package comparison and the other is single-package time breakdown; a task mentioning multiple packages and trends over time could blur the two, though the input shapes (array vs single name) make confusion less likely. |
downloads |
vulnerabilities |
low | Both take a single package name and are 'get info' style tools, but their subject matter (popularity vs security) is distinct enough that natural-language tasks rarely conflate them. |
package_info |
vulnerabilities |
low | A vague request like 'tell me about this package's risks' could nudge toward package_info (general info) vs vulnerabilities (security-specific), but the vulnerabilities description is specific enough to usually disambiguate. |
Compare the field