01Safety
0.0 / 30
What changed in the harness
Selection accuracy 95→95, token cost up 7%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.9 / 30
03Economics
20.0 / 20
04Discoverability
14.9 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
scanner_get_scan_status |
name_restates_behavior params_unexplained no_return_description |
Poll a scan you previously queued by passing its scan_id (the identifier returned when the scan was started). Returns the scan's current status — one of queued, running, completed, or failed — and the number of findings collected so far, so you can tell when the scan has finished. Keep polling until the status is completed before retrieving the findings. |
scanner_get_findings |
params_unexplained no_return_description |
Fetch the security findings of a scan that has reached completed status, passing the scan_id returned when you started or polled the scan. Returns a list of findings ordered by priority, each containing severity, CVE, template, title, URL, and priority_score. Only available after the scan has completed. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
scanner_register_target |
scanner_verify_target |
medium | Both belong to the same domain-ownership flow and both mention verification tokens/DNS records; a task like 'set up my domain for scanning' or 'verify my domain ownership' could plausibly land on register (which publishes the proof) vs verify (which checks it), so an agent may pick the wrong step first. |
scanner_register_target |
scanner_start_scan |
medium | For a not-yet-registered domain, 'scan my domain example.com' splits the agent: register matches the stated prerequisite ('you can only scan domains you own'), while start_scan matches the literal verb 'scan'. Both are defensible single-hop choices. |
scanner_get_scan_status |
scanner_get_findings |
medium | Both take a scan_id, and get_scan_status reports the finding count while get_findings lists details; a task like 'what did my scan turn up?' or 'show me my scan results' is ambiguous between the brief status/count and the detailed list, and the definitions alone don't force a choice. |
scanner_verify_target |
scanner_start_scan |
low | Both are target_id-based steps that follow registration, and phrasing like 'run a verification scan on my registered domain' or 'check my target' can push a shallow agent to verify_target when a scan (or vice versa) is intended. |
Compare the field