01Safety
0.0 / 30
What changed in the harness
Selection accuracy 93%, destructive-action safety rate 0% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.1 / 30
03Economics
19.2 / 20
04Discoverability
8.6 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
verity_search |
verity_research |
high | A vague research request like 'find out about X' could be routed to either: verity_search returns raw links/snippets while verity_research does search+scrape+summarize in one call. Both take a similar free-text query, and the distinction (raw results vs synthesized answer) is easy for an agent to miss. |
verity_research |
verity_alpha |
medium | Both are premium tools taking a 'query' string and producing synthesized intelligence. A crypto-related research question ('what's the outlook on X token?') could plausibly be sent to verity_research (general deep research) instead of verity_alpha (timing-aware strategic/crypto-specific synthesis), since the line between 'research' and 'actionable alpha' is a judgment call not encoded in the query itself. |
l402_balance |
l402_set_budget |
high | l402_set_budget's name strongly implies it sets/changes the budget, but it's actually read-only and just returns the configured startup cap — easily confused with l402_balance (remaining sats) when an agent is asked something like 'what's my budget?' or 'check my spending limit.' |
l402_balance |
l402_spending_report |
medium | A task like 'how much have I spent so far?' could trigger either: l402_balance reports remaining/spent totals in one line, while l402_spending_report gives the detailed per-domain/transaction audit. An agent might pick the simpler balance tool when the user actually wants the itemized report, or vice versa. |
l402_spending_report |
l402_set_budget |
low | Both are read-only l402 metadata tools returning budget-related info ('session' cap vs spending audit), so a generic 'show me my budget info' request could momentarily point at either, though their returned content is quite distinct once described. |
verity_summarize |
verity_sentiment |
low | Both take a 'text' parameter and are powered by Claude Haiku at similar pricing tiers, so a task like 'analyze this text' is mildly ambiguous, but their outputs (summary vs sentiment score/keywords) are different enough that confusion is unlikely once the specific ask ('summarize' vs 'sentiment/tone') is stated. |
Compare the field