01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 5%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.6 / 30
03Economics
20.0 / 20
04Discoverability
10.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
search_mcp_servers |
no_return_description |
Keyword and semantic search across every MCP server mcpindex indexes; use when the user knows the tool category they want but not which server. Returns a ranked list of up to `limit` matching servers, each with its slug, description, category, and quality score. |
compare_servers |
no_return_description |
Produces a side-by-side comparison of the 2-5 given MCP servers. Returns, for each server slug, its quality score, install path, and required environment variables so they can be evaluated against each other. |
assess_server |
no_return_description |
Runs an aggregated pre-flight trust assessment across all tools on an MCP server, for 'is THIS server worth integrating?' decisions. Returns the same advisory verdict object shape as check_tool_trust (directive, dimensions, freshness) computed in aggregate for the server; v1 is advisory only and may return UNVERIFIED if the server has not yet been probed. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
search_mcp_servers |
compare_servers |
low | search finds candidate servers by keyword while compare requires already-known slugs for 2-5 servers; a task like 'which server is better for X' could tempt an agent to call compare without first searching, but the description gap (unknown vs known server) makes true confusion unlikely. |
recommend_mcp_for_task |
search_mcp_servers |
medium | Both take a task/query describing what the user wants to do and return MCP servers; a vague request like 'find me an MCP server for reading PDFs' could plausibly go to either recommend (task-based ranking) or search (keyword/semantic search), since the line between 'task' and 'query' is blurry. |
compare_servers |
assess_server |
low | compare_servers needs multiple known slugs for side-by-side comparison, assess_server evaluates trust for a single server; a task like 'is server X good' is unambiguous for assess_server alone, so confusion is unlikely unless the agent mistakenly tries compare with one slug. |
search_mcp_servers |
assess_server |
low | search discovers servers by category/keyword while assess evaluates trust of one already-identified server; distinct enough purposes (discovery vs trust verdict) that natural-language tasks rarely conflate them. |
Compare the field