01Safety
30.0 / 30
What changed in the harness
Selection accuracy 75→75, token cost up 1%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
23.6 / 30
03Economics
16.2 / 20
04Discoverability
14.7 / 20
Highest-impact fix
Estimated gain +1 pointRewrite 4 underspecified tool descriptions
State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.
Description evidence
Defects and rewrites.
4 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
fetch_github_readme |
no_return_description |
Fetch and return the README content of a GitHub repository. Returns the README as markdown text. Best for: quickly reading a project's documentation from a repository URL (e.g., https://github.com/owner/repo). Not recommended for: Non-GitHub URLs — use free_extract instead. @readOnly true @idempotent true — makes outbound HTTP requests to raw.githubusercontent.com. |
fetch_csdn_article |
no_return_description |
Fetch and return the full content of a CSDN blog article as text. Best for: Chinese developer blog content on blog.csdn.net. Not recommended for: Other Chinese sites — use free_extract instead. @readOnly true @idempotent true — makes outbound HTTP requests to the CSDN article URL. |
fetch_juejin_article |
no_return_description |
Fetch and return the full content of a Juejin article as text, retrieved via the juejin.cn API. Best for: Chinese developer articles on Juejin. Not recommended for: Non-Juejin content — use free_extract instead. @readOnly true @idempotent true. |
search_with_synthesis |
params_unexplained |
Deep search with waterfall multi-engine verification. Returns structured results plus a prompt_hint for the agent to synthesize its own answer. No external LLM API key required; search and enrichment still make outbound network requests. Best for: Complex queries needing multi-source verification and LLM synthesis. Not recommended for: Simple fact-finding — use free_search instead. Parameters: query (search query), count (number of results to gather, 1-20), language (result language preference: auto, en, or zh), min_confidence (minimum source-reliability confidence 0-1; legacy values 2-3 are treated as min_source_count), min_source_count (minimum independent upstream provider families). @readOnly true @idempotent true — runs waterfall search across free+paid engines with content enrichment. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
free_search |
free_search_advanced |
medium | Both accept a plain query and differ only in optional filters/quality gates; a task like 'search for the most reliable X' or 'find recent X in 2026' doesn't clearly signal simple fact-finding vs filtered/high-confidence results, so an agent could route to either. |
free_search |
search_with_synthesis |
medium | Both take the same primary query input; free_search is quick fact-finding while search_with_synthesis targets complex multi-source verification with LLM synthesis. A task like 'search and confirm X across several sources' leaves the required depth a judgment call, making the wrong pick plausible. |
free_search_advanced |
search_with_synthesis |
medium | Their parameters (query, min_confidence, min_source_count, language, waterfall, enrichment) and behavior overlap heavily. A task wanting a 'thorough verified multi-engine search with high confidence only' fits both descriptions nearly equally, so selection is genuinely ambiguous. |
fetch_csdn_article |
fetch_juejin_article |
low | Both fetch a Chinese developer article from a URL and differ only by platform. A task phrased generically ('fetch this Chinese dev blog article') can be mapped to the wrong site-specific fetcher, though the URLs (blog.csdn.net vs juejin.cn) usually disambiguate on inspection. |
Compare the field