01Safety
30.0 / 30
What changed in the harness
Selection accuracy 98→98, token cost down 2%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.3 / 30
03Economics
18.5 / 20
04Discoverability
13.8 / 20
Highest-impact fix
Estimated gain +6 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
get_injuries |
no_return_description |
Track key player availability for the FIFA World Cup 2026. Returns a list of star players across all 48 teams with their current injury status (out, doubtful, recovering, or fit). Filter by team ID or FIFA code (e.g. 'usa', 'BRA') or by status; omit both filters to see all tracked players at once. |
Selection evidence
Confusable tool pairs.
12 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_teams |
get_team_profile |
medium | A generic task like 'tell me about the USA team' could route to either: get_teams returns rankings/details while get_team_profile returns the full profile, and both use the same team/group concepts. |
get_venues |
get_nearby_venues |
medium | 'What venues are around New York' or 'venues close to MetLife' could trigger get_venues (city filter) instead of get_nearby_venues (proximity), since the names differ only by 'nearby'. |
get_matches |
get_groups |
medium | 'What are the matches in group B?' fits both: get_matches has a group filter and get_groups returns each group's match schedule. |
get_matches |
get_bracket |
medium | 'Show me the round of 16 / knockout matches' is served by both get_matches (round filter) and get_bracket (knockout rounds with venues and dates). |
get_groups |
get_standings |
medium | 'Show me group A info/standings' — both accept a group letter; get_standings returns power rankings while get_groups returns teams and schedule, so group-level queries are genuinely ambiguous. |
get_odds |
get_standings |
high | Both cover group-by-group predictions derived from betting odds; get_standings explicitly cites betting odds and predictions, so 'who is expected to win group A' has no clear single winner. |
get_news |
get_injuries |
medium | 'Any injury updates for Brazil?' could route to get_news, which has an explicit 'injury' category, instead of the dedicated get_injuries tracker. |
get_historical_matchups |
compare_teams |
high | Both accept team_a/team_b and compare_teams explicitly advertises head-to-head history, so 'how has Argentina vs Brazil gone historically' could land on compare_teams rather than the dedicated get_historical_matchups. |
get_matches |
get_teams |
low | 'Which teams are playing in group A' fits both group filters — get_teams returns team details, get_matches returns team names per match — though primary intent usually disambiguates. |
get_groups |
get_schedule |
low | 'Show the schedule for group C' — get_groups embeds each group's match schedule while get_schedule covers the whole tournament, so schedule-producing queries can hit either. |
get_city_guide |
get_fan_zones |
low | 'What is there to do in Dallas during the Cup' could route to the city travel guide (things to do) or fan zones (activities/amenities), as both are host-city leisure resources. |
get_matches |
get_venues |
low | 'Which venues host matches in Mexico' — get_matches returns venue details per match and get_venues returns venues with notable events, so venue-focused match queries are mildly ambiguous. |
Compare the field