01Safety
30.0 / 30
What changed in the harness
Selection accuracy 97→100, token cost up 1%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.1 / 30
03Economics
20.0 / 20
04Discoverability
12.1 / 20
Highest-impact fix
Estimated gain +8 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_providers |
no_return_description |
List all tracked AI model providers (OpenAI, Anthropic, Google, etc.). Returns an array of providers, each with a short description and a link to its pricing page. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
search_models |
get_model_pricing |
high | search_models's own description says to use it for 'how much does <model> cost', yet get_model_pricing is the one that actually returns full pricing/blended cost/context window for a single model id — a query like 'what does GPT-5 cost' plausibly triggers either tool instead of the intended search-then-fetch pipeline. |
search_models |
cheapest_models |
medium | A request like 'find the cheapest coding model' could be served by search_models with category='coding' (returning models with prices for the agent to eyeball) or by cheapest_models with category='coding' and sort_by='input' — both take a category filter and surface prices, so the agent may not reliably pick the ranking-specific tool. |
get_model_pricing |
compare_models |
medium | For 'tell me the pricing for model A and model B' an agent might call get_model_pricing twice instead of using compare_models, since both ultimately surface per-model price fields and the task doesn't explicitly ask for a 'comparison' or 'verdict'. |
compare_models |
cheapest_models |
low | Both discuss ranking/cheapness, but compare_models strictly requires 2-5 explicit model_ids while cheapest_models operates over the whole catalog with no ids — the input shapes are different enough that confusion is unlikely except in vague prompts naming multiple specific models where either could seem to apply. |
search_models |
compare_models |
low | compare_models depends on ids obtained from search_models, making them a natural pipeline rather than competing choices for the same request; confusion is unlikely since compare_models' requirement for a model_ids array signals it needs prior lookup. |
Compare the field