01Safety
30.0 / 30
What changed in the harness
Selection accuracy 98→98, token cost up 3%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.8 / 30
03Economics
20.0 / 20
04Discoverability
18.0 / 20
Highest-impact fix
Estimated gain +2 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
scan_directory |
no_return_description |
Scan a local MCP server directory for backdoors, exfiltration code, obfuscation, and dangerous patterns. Use this for MCP servers installed locally or cloned from GitHub. Returns the results of the scan indicating which risk categories were detected in the directory. |
check_prompt_injection |
no_return_description |
Scan text (tool descriptions, tool responses, MCP server manifest content) for prompt injection attacks, hidden instructions, zero-width characters, jailbreak patterns, and instruction overrides. Returns a report of any prompt injection indicators and hidden content found in the scanned text. |
Selection evidence
Confusable tool pairs.
1 pair where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
scan_package |
scan_directory |
medium | Both tools scan MCP servers for the exact same threat categories (backdoors, exfiltration, obfuscation), so a natural-language task phrased generically as 'scan this MCP server for security issues' or 'check this MCP server for backdoors' could cause the agent to pick the wrong member if the source (npm package vs. local path) is not explicitly stated. The descriptions do disambiguate on input type (package_name with npm hint vs. path), so confusion is plausible but avoidable with careful reading. |
Compare the field