01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100%, destructive-action safety rate 100% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.9 / 30
03Economics
19.7 / 20
04Discoverability
16.9 / 20
Highest-impact fix
Estimated gain +3 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
sthan_verify_address |
sthan_parse_address |
medium | Both accept a freeform US address string and return a standardized, normalized form. A task phrased as 'clean/standardize this address' or 'break down this address' lacks a deliverability check and is genuinely ambiguous between field-splitting and validation; only a task explicitly mentioning deliverability or 'real/valid' points to verify. |
sthan_autocomplete_address |
sthan_autocomplete_city |
medium | Both take a partial 'text' input and return US suggestions with state. A type-ahead task like 'suggest as user types San' is ambiguous: the text could be a city name or the start of a street address, and the definitions don't disambiguate bare partial text without knowing which form field it belongs to. |
sthan_autocomplete_address |
sthan_autocomplete_zipcode |
medium | Both are partial-text autocomplete with near-identical input shape. Numeric partial input such as '123' or '9021' could plausibly be a street number or a ZIP code; the choice only becomes clear if the agent knows which form field the user is typing in, which a natural-language task often omits. |
sthan_verify_address |
sthan_autocomplete_address |
low | A task with a partial/incomplete address could plausibly mean 'complete what I'm typing' (autocomplete) or 'check whether my partial address resolves to something deliverable' (verify). The definitions do separate them, but only a task explicitly about confirming deliverability reliably selects verify. |
sthan_geocode |
sthan_reverse_geocode |
low | A task phrased generically as 'geocode this' is ambiguous about direction, but the required input schemas (address string vs latitude/longitude) and each description's explicit 'to go the other way use...' cross-reference make wrong selection unlikely for well-specified tasks. |
sthan_autocomplete_city |
sthan_autocomplete_zipcode |
low | Both are 'type-ahead returning US place + state' tools. Input format usually separates them (a city name vs digits), but a bare task like 'autocomplete this location' without the text sample is genuinely ambiguous between the two. |
Compare the field