01Safety
0.0 / 30
What changed in the harness
Selection accuracy 95→98, token cost up 4%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.8 / 30
03Economics
20.0 / 20
04Discoverability
13.1 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
scan_safety |
no_return_description |
Run a focused safety scan on an AI skill or MCP server, checking for prompt injection, shell execution, network exfiltration, credential theft, obfuscated payloads, public network binding (0.0.0.0), and risky npm lifecycle scripts (preinstall/install/postinstall). Accepts any input format (owner/repo, npm package, Smithery URL, etc.). Returns a safety report listing the findings for each checked category, including any detected risks, their severity, and the supporting details. |
list_packages |
no_return_description |
Browse curated, pre-scored AI skill packages organized by use case. Each package contains vetted skills with trust scores. Available packages: Claude Power User, Full-Stack Vibe Coder, Data & Research, DevOps & Infrastructure, Content & Marketing. If package_name is omitted, returns the list of available packages; if a package_name is given, returns that package's vetted skills and their trust scores. |
watch_repo |
no_return_description |
Start monitoring a repo or package for trust score changes. Alerts when the score changes significantly (±0.3 points or tier change). Requires a paid API key. Returns a confirmation that monitoring has been started for the given repo and email address. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
check_watched |
batch_check |
low | Both are 'check repos' tools, so a task like 'batch-check the trust scores of my repos' could pull the agent toward batch_check. But check_watched uniquely mentions 'watched repos' and requires an email, while batch_check takes an explicit repo array, so a well-matching agent is unlikely to genuinely swap them. |
check_trust_score |
batch_check |
medium | A task like 'check the trust scores of these few repos' is genuinely ambiguous: check_trust_score scores any single repo, batch_check handles up to 5 in one call. Without an explicit count or a Pro/Team key consideration, the agent may pick the wrong one, either looping the single-repo tool or choosing batch when only one repo was meant. |
check_trust_score |
check_watched |
medium | Both return trust scores and share the 'check/score' tokens. A task like 'what are the current trust scores for the repos I follow' could route the agent to check_trust_score (which needs each repo passed individually) instead of the correct check_watched (which rescans watched repos from an email); only the 'watched' keyword and email input disambiguate. |
Compare the field