01Safety
0.0 / 30
What changed in the harness
Selection accuracy 93→88, token cost up 6%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.6 / 30
03Economics
20.0 / 20
04Discoverability
13.1 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
verify_baseline |
no_return_description |
Compares a fresh scan of MCP config JSON against a recorded baseline and returns a verification result — drift status, added/removed/changed servers, and any rug-pull signals detected. If no baseline is supplied, one is built from the current config and returned instead for first-time approval. Static; never executes the server. |
generate_agent_rule |
no_return_description |
Generates and returns the CallLint agent-safety rule text as a string for the given host (one of: claude, agents, cursor, copilot, codex, gemini, windsurf, cline, command), ready to paste into that host's rules file. |
generate_ci_gate_snippet |
no_return_description |
Generates and returns the contents of a GitHub Actions workflow file (.github/workflows/calllint.yml) as a string, for gating a repo on its agent-tool surface. In mode=drift (the default) the workflow fails CI on approved-state drift; in mode=scan-all it only reports findings without failing the build. |
Selection evidence
Confusable tool pairs.
1 pair where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
scan_mcp_config_path |
scan_mcp_config_json |
medium | Both scan an MCP config and return verdicts; an agent given a task like 'scan this MCP config' without specifying whether it has a file path or raw JSON text in hand could pick the wrong tool, especially if the config content is quoted inline but originates from a file. |
Compare the field