01Safety
30.0 / 30
What changed in the harness
Selection accuracy 95→95, token cost up 2%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.8 / 30
03Economics
20.0 / 20
04Discoverability
15.8 / 20
Highest-impact fix
Estimated gain +1 pointRewrite 6 underspecified tool descriptions
State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.
Description evidence
Defects and rewrites.
6 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
lookup_ip_address_details |
no_return_description |
Look up an IPv4 or IPv6 address (or the caller's own IP if omitted) and return a full JSON profile: geolocation, network/ASN details, privacy/security flags (VPN, proxy, Tor, hosting), company/organization data, and abuse contact information. |
lookup_ip_address_location |
no_return_description |
Look up an IPv4 or IPv6 address (or the caller's own IP if omitted) and return its geographic location data: country, region, city, latitude/longitude coordinates, timezone, and postal code. |
lookup_ip_address_privacy |
no_return_description |
Look up an IPv4 or IPv6 address (or the caller's own IP if omitted) and return privacy/security indicators for it, including whether it is a VPN, proxy, Tor exit node, or hosting/datacenter IP, plus any known abuse signals. |
lookup_ip_address_network |
no_return_description |
Look up an IPv4 or IPv6 address (or the caller's own IP if omitted) and return its network/ASN details: the Autonomous System Number, network operator name, announced route, and regional internet registry (RIR). |
lookup_ip_address_company |
no_return_description |
Look up an IPv4 or IPv6 address (or the caller's own IP if omitted) and return the company/organization associated with it, including company name, domain, and organization type (e.g. ISP, hosting, business). |
lookup_ip_address_abuse_contacts |
no_return_description |
Look up an IPv4 or IPv6 address (or the caller's own IP if omitted) and return abuse contact details for reporting malicious activity, including the network administrator's email, phone number, and physical address. |
Selection evidence
Confusable tool pairs.
10 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
lookup_ip_address_details |
lookup_ip_address_company |
medium | details already includes company data per its description, so a request like 'get company info for this IP' could plausibly route to the comprehensive details tool instead of the dedicated company tool. |
lookup_ip_address_details |
lookup_ip_address_location |
medium | details includes geolocation, so a task like 'where is this IP located' might be satisfied by either the location-specific tool or the all-in-one details tool. |
lookup_ip_address_details |
lookup_ip_address_network |
medium | details includes network details, so a request for ASN/network operator info could be routed to either the network-specific tool or the comprehensive details tool. |
lookup_ip_address_details |
lookup_ip_address_privacy |
medium | details includes privacy/security info, so a request like 'check if this IP is a VPN' could plausibly go to either the privacy-specific tool or the comprehensive details tool. |
lookup_ip_address_location |
lookup_ip_address_company |
low | Both are IP lookup tools with distinct, non-overlapping domains (geography vs company); little realistic confusion beyond generic IP lookup phrasing. |
lookup_ip_address_location |
lookup_ip_address_network |
low | Location and network/ASN data are conceptually distinct enough that most natural-language requests would clearly indicate one or the other. |
lookup_ip_address_location |
lookup_ip_address_privacy |
low | Location and privacy/security data serve different intents; minimal overlap beyond shared IP-lookup framing. |
lookup_ip_address_network |
lookup_ip_address_company |
medium | Network operator and company/organization info can overlap conceptually (e.g. 'who owns this IP' could mean ASN operator or company), causing plausible selection ambiguity. |
lookup_ip_address_privacy |
lookup_ip_address_company |
low | Privacy/security (VPN, proxy, abuse) and company/org data are distinct concerns with little natural-language overlap. |
lookup_ip_address_privacy |
lookup_ip_address_network |
low | Privacy/security detection and ASN/network registry data address different questions, so confusion is unlikely despite shared 'IP lookup' framing. |
Compare the field