01Safety
30.0 / 30
What changed in the harness
Selection accuracy 93→95, token cost up 1%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
24.9 / 30
03Economics
15.5 / 20
04Discoverability
17.8 / 20
Highest-impact fix
Estimated gain +1 pointRewrite 8 underspecified tool descriptions
State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.
Description evidence
Defects and rewrites.
8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_categories |
no_return_description |
Lists all developer infrastructure deal categories (e.g. databases, cloud hosting, CI/CD, monitoring, auth, search) along with how many offers each contains. Call this first to see what's available before searching. |
search_offers |
no_return_description |
Searches 1,500+ developer tool offers (free tiers, credits, discounts) from vendors like AWS, Vercel, Supabase, and Cloudflare, returning matching deal records with vendor, pricing, and eligibility details. Filter by keyword, category, or eligibility type (public, accelerator, oss, student, fintech, geographic, enterprise), and sort by vendor, category, or recency. |
get_offer_details |
no_return_description |
Returns full pricing and free-tier details for a specific vendor's deal. Set include_alternatives to true to also return full deal objects for up to 5 similar vendors in the same category — useful for recommending alternatives or evaluating options. |
get_deal_changes |
no_return_description |
Returns a list of recent pricing/free-tier change events (removals, limit reductions, limit increases, new free tiers, restructures) affecting developer tools, including vendor, change type, and date, filterable by date, change type, or vendor(s). Use when advising on vendor lock-in risk or staying current on pricing shifts. |
get_stack_recommendation |
no_return_description |
Returns a curated free-tier infrastructure stack (hosting, database, auth, and more) with specific vendor picks per category, tailored to your described project and any required capabilities. Covers SaaS apps, API backends, static sites, mobile apps, AI/ML projects, e-commerce, and DevOps. |
estimate_costs |
no_return_description |
Returns a per-service cost breakdown for the vendors you specify at a given scale (hobby/startup/growth), including free tier limits, estimated cost, free alternatives, and warnings about recent pricing changes. Use during project planning, code reviews, or deployment setup. |
get_expiring_deals |
no_return_description |
Returns a list of developer tool deals, free tiers, or credits expiring within the given number of days ahead, including vendor, category, and expiration date. Use to avoid service disruptions and find replacements before deadlines. |
get_weekly_digest |
no_return_description |
Returns a curated summary of the past week's developer tool pricing changes, new offers, and upcoming deadlines, falling back to a 30-day window if fewer than 3 changes occurred in the past week. Use for regular check-ins on what's changed in developer pricing. |
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_deal_changes |
get_expiring_deals |
medium | A vague request like 'what deals should I watch out for' could map to either tracking pricing/tier changes (get_deal_changes) or upcoming expirations (get_expiring_deals) since both frame risk around existing deals. |
get_newest_deals |
get_deal_changes |
medium | Both answer 'what's new with deals' queries, but get_newest_deals covers newly added/verified deals while get_deal_changes covers pricing/tier changes to existing ones — an agent given 'show me recent deal updates' could pick either. |
get_newest_deals |
get_weekly_digest |
medium | Both serve periodic 'what's new' check-in use cases; a request for 'a summary of new deals this week' could plausibly route to either the raw newest-deals list or the curated weekly digest. |
get_stack_recommendation |
audit_stack |
high | Both center on 'your infrastructure stack' — a request like 'help me improve my stack' is ambiguous between getting a fresh recommended stack (get_stack_recommendation) and auditing an existing one for savings/risks (audit_stack), especially since audit_stack requires listing services which the user may or may not mention. |
search_offers |
get_new_offers |
medium | A request like 'find developer tool deals' doesn't specify whether the user wants a general filtered search (search_offers, which also supports sort=newest) or specifically recently-added offers (get_new_offers), creating overlap especially since search_offers can already sort by newest. |
get_new_offers |
get_deal_changes |
medium | Both are 'check recent activity' tools; a query like 'what changed with developer tool pricing lately' could be routed to get_new_offers (new listings) or get_deal_changes (pricing/tier modifications) depending on interpretation. |
Compare the field