01Safety
0.0 / 30
Public leaderboard
Public assessment
Thezenmonster/agentscore-mcp-server (@agentscore-xyz/mcp-server)
What changed in the harness
Selection accuracy 100→100, token cost up 2%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.2 / 30
03Economics
20.0 / 20
04Discoverability
13.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
4 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
get_verdict |
no_return_description |
Get a trust verdict for an npm package. Returns one of 'allow', 'warn', or 'block', derived from the package's scan findings (score and severity), together with its monitoring status and publisher posture. Use this before installing or connecting to an MCP server. |
monitor_status |
no_return_description |
Check whether an npm package is under continuous monitoring. Returns the package's monitoring status, its scan history, current score, risk level, and recent changes. |
check_my_repo |
no_return_description |
Inspect the current repo for MCP dependencies and look up an AgentScore verdict for each detected package. Returns a summary listing each package's verdict and which packages should be gated in CI. Use this when a developer wants to understand all MCP packages in a repo instead of scanning one package at a time. |
install_policy_gate |
no_return_description |
Install the AgentScore Policy Gate into this repo by writing .github/workflows/agentscore-policy-gate.yml with OIDC authentication (no API key needed). Yields the resulting workflow file, which detects MCP dependencies in the repo and includes them so the gate can enforce checks on CI; the gate auto-provisions the repo on first push. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
generate_policy_gate_setup |
install_policy_gate |
high | A task like 'set up the AgentScore policy gate in my repo' or 'generate the policy gate setup' matches both: one returns the OIDC YAML needed for setup, the other writes it into .github/workflows. The agent must guess whether the user wants the workflow file generated/returned or actually installed. |
check_exposure |
check_abuse |
medium | Both start with 'check' and take a single package/agent name. A vague task like 'check package axios — is it a problem?' or 'investigate this package during the incident' could route to blast-radius exposure or abuse-database lookup, since neither description fully disambiguates a generic 'check this package' request. |
check_abuse |
check_my_repo |
low | A repo-wide task like 'check if the packages in my repo are abusive or flagged' could plausibly route to per-item check_abuse instead of the repo-scanning check_my_repo, but the descriptions mostly disambiguate by scope (single agent/package vs whole repo with verdict summaries). |
Compare the field