01Safety
0.0 / 30
What changed in the harness
Selection accuracy 98→95, token cost up 2%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.8 / 30
03Economics
17.2 / 20
04Discoverability
11.7 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
analyze_award_patterns |
params_unexplained |
Analyze how federal contracts in a NAICS industry are typically awarded: award-size distribution, competition mix (full & open / limited / sole-source), bid intensity (single-bid rate, expected bidders), contract vehicles, and pricing types, from public USAspending + FPDS data at a cost of 3 credits. Pass the required naicsCode — the North American Industry Classification System code identifying the industry to analyze (e.g. '541511') — and optionally narrow the analysis with pscCode (product/service code), years (lookback period in years, default ~3), or state (2-letter place-of-performance filter). |
analyze_market |
params_unexplained |
Produce a full market-intelligence snapshot for a NAICS industry: 5-year federal spending trend, industry size (Census), top contractors with market share and HHI concentration, set-aside distribution, geographic distribution, and contract-vehicle usage, from public USAspending + Census data at a cost of 5 credits. Pass the required naicsCode — the North American Industry Classification System code identifying the market to analyze (e.g. '541511') — and optionally pass pscCode for sharper competitor filtering by product/service code or setAside for program-specific set-aside context. |
Selection evidence
Confusable tool pairs.
1 pair where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
analyze_award_patterns |
analyze_market |
medium | Both tools ingest the same required inputs (naicsCode + optional pscCode), both draw on the same public award/market datasets, and both expose overlapping output dimensions (contract-vehicle usage, distribution, competition mix). At high-credit stakes (3 vs 5 credits), a vague task like 'analyze the market for NAICS 541611 and how awards are structured' could plausibly route to either tool since the names share 'analyze' and neither description crisply anchors ownership of the full-snapshot versus award-dynamics scope. |
Compare the field