01Safety
30.0 / 30
What changed in the harness
Selection accuracy 96→96, token cost up 8%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.6 / 30
03Economics
19.9 / 20
04Discoverability
12.0 / 20
Highest-impact fix
Estimated gain +8 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
search_countries |
params_unexplained |
Search countries by partial name match (e.g. "arg" finds Argentina) and return the matching countries as a paginated list. Use the required q for the search term; optionally set page (starting at 1) to select a page and limit (1–100) to control results per page. |
get_states |
name_restates_behavior params_unexplained no_return_description |
List the states/provinces of a country as a paginated result set, using the required ISO2 or ISO3 country_code. Optionally paginate with page and limit (1–100 per page), and choose a fields detail level (basic, standard, or full) to control the data included in each returned state record. Requires Starter plan or above. |
get_cities |
name_restates_behavior params_unexplained no_return_description |
List the cities belonging to a state/province, identified by the required numeric state_id (obtained from a get_states response), and return them as a paginated result set. Optionally set page and limit (1–100 per page) to control pagination and fields (basic, standard, or full) to control the detail level of each returned city. Requires Starter plan or above. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_states |
get_cities |
medium | Hierarchical lookup: a task like 'list all cities in Argentina' may make the agent call get_cities directly, but it needs a state_id that only comes from get_states; conversely a states request could be answered with get_cities. The dependency between the two invites choosing the wrong member. |
list_countries |
search_countries |
low | Both return paginated country lists and overlap in params; loose phrasing like 'find a list of all countries' or 'get me the countries' maps ambiguously to either, though the list/search verbs usually disambiguate. |
search_countries |
global_search |
medium | Both accept a free-text search term and return matches; a task like 'search for Buenos Aires' or 'find Pampas' requires global_search (covers states/cities), but an agent may wrongly pick search_countries, or over-reach to global_search for a country-only query. |
get_country |
get_currency_rate |
medium | Both are keyed by country_code and overlap in output: get_country's description advertises the live USD exchange rate, so a currency-rate request may wrongly select get_country, and a country-data request may wrongly select get_currency_rate. |
Compare the field