01Safety
0.0 / 30
What changed in the harness
Selection accuracy 96→98, token cost up 6%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.7 / 30
03Economics
20.0 / 20
04Discoverability
12.9 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
4 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
volt_recommend_route |
no_return_description |
Get the optimal provider recommendation for a model based on cost, latency, reliability, or balanced optimization, filtered by quality/latency thresholds and blocked providers. Returns the recommended provider/offering with its cost, latency, and quality metrics, plus an estimated savings figure vs. current_cost_per_million if provided. |
volt_get_spend |
no_return_description |
Get spending summary by provider and model for today, 7 days, or 30 days. Returns total spend and a breakdown of spend per provider and per model over the requested time_range. |
volt_get_savings |
no_return_description |
Compare actual spend against optimal routing over the given time_range. Returns actual spend, the optimal spend if requests had been routed ideally, savings already achieved, and savings missed by not routing optimally. |
volt_set_budget_alert |
name_restates_behavior no_return_description |
Configure a spend threshold that triggers an alert when spend for the given period (daily, weekly, or monthly) exceeds it. Returns confirmation of the created budget alert, including its threshold and period. |
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
volt_check_price |
volt_recommend_route |
high | Both take a 'model' parameter and surface provider/pricing comparisons; a request like 'what's the cheapest provider for gpt-4o' could plausibly route to either check_price (sorted price list) or recommend_route (optimized recommendation), especially since an agent may not realize recommend_route also factors in latency/reliability/current cost. |
volt_get_spend |
volt_get_savings |
medium | Both are time_range-scoped summary tools; a vague request like 'show me how we're doing on costs this week' could be routed to either the raw spend summary or the savings-vs-optimal comparison, since the distinction between 'spend' and 'savings' isn't always clear from a casual phrasing. |
volt_recommend_route |
volt_get_savings |
medium | Both surface 'savings' concepts — recommend_route computes savings vs a user-supplied current cost for a specific model, while get_savings reports historical savings achieved/missed from actual routing decisions; a query like 'how much am I saving' is ambiguous between a forward-looking recommendation and a backward-looking report. |
volt_check_price |
volt_get_savings |
low | Both involve price/cost comparison across providers, but check_price is a stateless pricing lookup while get_savings requires historical spend data; confusion is possible only in loosely worded 'compare costs' requests but is less likely given differing required inputs. |
volt_check_price |
volt_get_spend |
low | Shared tokens are superficial (model/provider); check_price is forward-looking pricing comparison and get_spend is historical usage reporting, so genuine task confusion is unlikely except in very vague 'show me costs' phrasing. |
volt_recommend_route |
volt_get_spend |
low | recommend_route is a provider recommendation engine while get_spend is a historical spend report; little semantic overlap beyond both being volt cost-related tools, so confusion is unlikely for a clear task. |
Compare the field