01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.6 / 30
03Economics
19.2 / 20
04Discoverability
10.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
submit_review |
no_return_description |
Record your real-world integration experience for an API after a successful or failed integration. Submit a 1-5 rating, title, detailed body, CLI headless-experience score, setup difficulty, docs quality, a recommend flag, and optional structured integration report. Returns a confirmation that the review was saved and now feeds the score the next agent sees. Takes ~10 seconds and is required after any integration sourced via discover_apis or recommend. Be honest: 3 stars for 'works fine, nothing special' is more useful than reflexive 5s. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_api_details |
get_api_docs |
high | Both take an API slug and return API-specific info, so a task like 'get me everything about stripe-api' or 'show the API info so I can integrate it' could select either; 'details' and 'docs' are near-synonyms in the agent's view. |
discover_apis |
compare_apis |
medium | A loosely-phrased task like 'compare the best SMS APIs' could make the agent call compare_apis (which requires specific slugs the user didn't give) or discover_apis, which already returns ranked options with pricing and scores, since these are the two 'comparison-flavored' tools. |
get_api_details |
get_reviews |
medium | Both accept the same slug parameter and surface rating/score data (details has scores and quality breakdown; reviews has ratings and CLI scores), so 'what's the rating for openai-api?' or 'how is this API rated?' is genuinely ambiguous between the two. |
get_api_docs |
get_reviews |
low | Both descriptions advertise 'integration' data for the same slug (docs covers quickstart and gotchas before coding; reviews covers structured integration reports and challenges), so 'get the integration info for stripe-api' could plausibly pick either, though the how-to vs experience intent usually separates them. |
Compare the field