01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 14%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
24.6 / 30
03Economics
19.7 / 20
04Discoverability
16.8 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
7 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
http_request |
params_unexplained no_return_description |
Sends an HTTP request and returns the response (status, headers, and body). Parameters: url (target endpoint), method (HTTP verb), headers (custom request headers), body (raw request payload), query_params (URL query string parameters), timeout (max time in ms before aborting), auth (authentication settings such as basic, bearer, oauth2, api_key, or custom), retry (whether to retry on failure), max_retries (max retry attempts), retry_delay (ms to wait between retries). |
list_collections |
name_restates_behavior no_return_description |
Returns the names of all saved request collections currently stored, so a collection name can be chosen for use with list_requests or request_from_collection. |
list_requests |
name_restates_behavior params_unexplained no_return_description |
Returns the names and details of all saved requests within the given collection. Parameter: collection_name identifies which saved collection to look up. |
save_request |
name_restates_behavior params_unexplained no_return_description |
Stores an HTTP request definition under a named collection so it can be retrieved and replayed later via request_from_collection. Parameters: collection_name (collection to save into, created if missing), request_name (identifier for the saved request), request (the HTTP request definition, including url, method, headers, body, query_params, timeout, and auth), description (optional notes about the request's purpose). Returns confirmation that the request was saved. |
request_from_collection |
params_unexplained no_return_description |
Retrieves a saved request from a collection and, unless disabled, executes it, returning the request definition and (if executed) the HTTP response. Parameters: collection_name (collection containing the request), request_name (name of the saved request to retrieve), environment_name (environment whose variables to substitute into the request), execute (whether to actually send the request rather than just returning its definition; defaults to true). |
set_environment_variable |
name_restates_behavior params_unexplained no_return_description |
Creates or updates a variable in a named environment so it can be substituted into future HTTP requests. Parameters: name (variable name), value (variable value), is_secret (whether to mask the value in output/logs), environment_name (environment to store the variable in; defaults to the active environment). Returns confirmation that the variable was set. |
get_environment_variables |
params_unexplained no_return_description |
Returns all variable names and values currently defined in the specified environment (or the active environment if omitted). Parameter: environment_name identifies which environment to read from. |
Selection evidence
Confusable tool pairs.
7 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
set_environment_variable |
get_environment_variables |
low | Both target environment variables but one sets and one reads; a request like 'check the environment variable' is clearly retrieval while 'update the environment variable' is clearly setting, so intent rarely overlaps despite shared naming. |
list_collections |
list_requests |
medium | A vague task like 'show me what's available' could be misrouted since list_collections lists collection names while list_requests lists requests within one collection; an agent unsure of scope might call the wrong one, especially if collection_name isn't yet known. |
http_request |
request_from_collection |
medium | A task like 'run the login request' could be satisfied by either directly firing http_request with hardcoded details or by request_from_collection if the request was previously saved, causing ambiguity about which mechanism to use for 'send this named request'. |
http_request |
save_request |
low | save_request persists a request rather than executing it; a task to 'send and store this request for later' could tempt an agent to only call http_request when save_request was also needed, but the actions are functionally distinct (execute vs persist) making true selection confusion less likely. |
list_requests |
request_from_collection |
low | Both operate on a named collection, but list_requests enumerates request names while request_from_collection fetches/executes one specific request; a task like 'get the requests in my collection' is unambiguous, though 'get me that request' could momentarily conflate the two. |
list_requests |
save_request |
low | Both require collection_name but serve opposite purposes (read vs write); confusion is unlikely except in poorly specified tasks like 'update my collection list' which doesn't map cleanly to either. |
save_request |
request_from_collection |
low | Both take collection_name and request_name, but one saves a new request and the other retrieves/executes an existing one; a task like 'add this request to my collection and run it' might cause sequencing confusion but not outright tool-choice ambiguity. |
Compare the field