01Safety
30.0 / 30
What changed in the harness
Selection accuracy 93→95, token cost up 9%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.6 / 30
03Economics
20.0 / 20
04Discoverability
18.2 / 20
Highest-impact fix
Estimated gain +1 pointRewrite 3 underspecified tool descriptions
State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
scan |
no_return_description |
Discover all MCP server configs and test their connections via JSON-RPC handshake. Returns a list of configured servers with each one's connection status (success/failure) and any handshake error details. |
security |
no_return_description |
Audit all MCP server configs for security issues like hardcoded secrets, tokens in args, and shell injection patterns. Returns a list of findings per server, each describing the issue type, severity, and location in the config. |
doctor |
no_return_description |
Run all checks at once: scan connections, audit security, and benchmark latency for all configured MCP servers. Returns a combined report with connection status, security findings, and latency measurements for each server. |
Selection evidence
Confusable tool pairs.
1 pair where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
bench |
doctor |
medium | Both take no parameters and cover latency benchmarking for all configured MCP servers; a task phrased like 'check latency of my MCP servers' could plausibly route to either 'bench' (matches literally) or 'doctor' (which also includes benchmarking as part of a broader health check), causing an agent to pick the superset tool when only latency was wanted, or vice versa if it conflates 'doctor' with a narrower diagnostic. |
Compare the field