01Safety
30.0 / 30
What changed in the harness
Selection accuracy 93→95, token cost down 0%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.1 / 30
03Economics
13.1 / 20
04Discoverability
11.1 / 20
Highest-impact fix
Estimated gain +9 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
estimate_cost |
params_unexplained |
Preview how many credits one or more planned tool calls would cost before making them, with no credits spent. The 'calls' parameter is an array of up to 50 objects, each with 'tool' (the exact name of a tool on this server, e.g. get_event, batch_get_events, get_odds) and 'arguments' (the exact arguments you'd pass to that tool, e.g. {"event_id": 123, "include_odds": true}). Because costs are data-dependent (odds/intelligence/splits not yet ingested for an event are free, and batch_get_events ids that don't exist cost nothing), each call is returned as a [min_credits, max_credits] range rather than a single number. |
Selection evidence
Confusable tool pairs.
7 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_odds |
get_odds_history |
medium | A task phrased as 'what are the odds for this game' matches both tool names, and only the descriptions' cross-reference (current line vs line movement over time) separates them; a request like 'how have the odds moved' can be misrouted to get_odds, while a bare 'odds' request risks grabbing the history variant. Both take event_id. |
list_sports |
list_seasons |
medium | Tasks like 'what season is it for MLB' or 'list available seasons/leagues' match both: list_sports embeds each league's current_season while list_seasons returns active seasons with phase/dates. Only the cross-referenced descriptions disambiguate, so a vague 'current season' or 'seasons' request could hit either. |
get_stats |
get_splits |
medium | The word 'splits' is genuinely ambiguous in natural language: get_splits is public betting splits (bets%/handle%) while get_stats returns home/away performance splits and boxscore rates. 'Show me the splits for this game' could route to either, and both accept only event_id. |
list_events |
query_events |
medium | Both return events and the choice depends entirely on task phrasing: free-text requests like 'live nfl games today' target query_events, while structured filters target list_events, and the descriptions themselves label query_events as the natural-language route. A vague 'find events' gives no signal, risking the wrong pick. |
get_event |
batch_get_events |
low | Descriptions explicitly split single id vs multiple ids, but cardinality is the only cue; a task like 'fetch events 5, 6, 7' could fire batch for a single id or get_event repeatedly if the agent is lax on count. |
get_event |
get_odds |
low | get_event can inline odds via include_odds while get_odds returns only the line, so 'get current odds for event X' has two plausible-looking tools; picking get_event without include_odds yields no odds, or picking get_odds misses that event detail was also wanted. |
list_teams |
get_team |
low | Both handle 'team' lookups: list_teams is for resolving/searching ids (q, league, etc.) while get_team returns a single profile by id. A task like 'get the Lakers details' could prompt get_team directly without first resolving the id via list_teams, or list_teams when a single team's profile was wanted. |
Compare the field