01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 17%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
24.6 / 30
03Economics
19.7 / 20
04Discoverability
13.8 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
7 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
http_request |
params_unexplained no_return_description |
Executes an HTTP request against a target endpoint. `url` is the full request URI; `method` selects the HTTP verb (GET, POST, PUT, etc.). `headers` sets extra request headers, `body` is the raw payload string (e.g. JSON) sent with the request, and `query_params` appends each key/value pair as a query string parameter. `timeout` limits the request duration in milliseconds. `auth` supplies credentials via one of the supported types (basic, bearer, oauth2, api_key, custom) and their related fields to construct the appropriate authorization headers or query parameter; omit it to send an unauthenticated request. When `retry` is true, failed requests are retried up to `max_retries` times (default 3) with `retry_delay` milliseconds (default 1000) between attempts. Returns the HTTP response, including status code, response headers, and body. |
list_collections |
name_restates_behavior no_return_description |
Returns every request collection currently stored in Postmancer, including each collection's name, so callers can discover what is available before saving new requests or inspecting existing ones with list_requests. |
list_requests |
name_restates_behavior params_unexplained no_return_description |
Returns the requests stored in a given collection, including each saved request's name and configuration, so callers can inspect or reference them before retrieval. `collection_name` is the exact name of an existing collection to inspect. Returns a list of the saved requests within that collection. |
save_request |
name_restates_behavior params_unexplained no_return_description |
Stores a complete HTTP request definition under a named collection so it can be retrieved or executed later via request_from_collection. `collection_name` is the target collection (an existing one is updated, or a new one is created); `request_name` is the unique identifier to store the request under; `request` is the full request spec including url, method, headers, body, query_params, timeout, and optional auth; `description` is an optional human-readable note about the saved request. Returns confirmation that the request was saved, identifying the collection and request name. |
request_from_collection |
params_unexplained no_return_description |
Loads a previously saved request and optionally executes it. `collection_name` and `request_name` identify which saved request to fetch. `environment_name` optionally selects an environment whose variables are substituted into the request's url, headers, body, and auth values; when omitted the default environment is used. `execute` (default true) controls whether the request is actually sent: when true, returns the live HTTP response (status, headers, body); when false, returns only the resolved request definition without sending it. |
set_environment_variable |
name_restates_behavior params_unexplained no_return_description |
Defines or updates an environment variable that can be referenced inside HTTP request fields so values can vary per environment. `name` is the variable key used in request templates and `value` is its string value. Set `is_secret` to true to mask the value in logs and output. `environment_name` selects which environment to write to; when omitted the default environment is targeted. Returns confirmation that the variable was saved, including the affected environment and variable name. |
get_environment_variables |
params_unexplained no_return_description |
Reads all environment variables defined in a specific environment. `environment_name` selects which environment to read from; when omitted the default environment is used. Returns the environment's variables (names, values, and whether each is marked as a secret). |
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
list_collections |
list_requests |
medium | A user asking to 'list all my requests' gives no collection name; since list_requests requires collection_name, an agent may pick list_collections to enumerate top-level items, and 'show all requests/collections' is genuinely ambiguous about which level to list |
http_request |
request_from_collection |
medium | Both tools send/execute HTTP requests. If the user says 'run the login request' or 'execute request X' referring to a saved request, the agent may build the request manually via http_request instead of retrieving and executing it via request_from_collection, since both satisfy a request-execution task |
list_requests |
request_from_collection |
medium | Both take collection_name and the word 'get' appears in request_from_collection's description ('Retrieves...'). A task like 'get the requests from collection X' or 'retrieve request auth from collection Y' could be mapped to listing versus retrieving/executing a single named request |
set_environment_variable |
get_environment_variables |
low | Set vs get are usually explicit, but both operate on 'environment variables' and a terse task like 'handle environment variables for staging' or 'what's set in the environment' lacks a direction, making the wrong member possible |
http_request |
list_requests |
low | 'Request' is overloaded: a task such as 'show me my requests' or 'run the specified request' could be read as sending an HTTP request versus listing saved request entries, though most phrasing with send/show disambiguates |
save_request |
request_from_collection |
low | Both operate on a named request within a named collection; a task like 'get request X from collection Y and keep it usable' or 'handle request X in collection Y' lacks a clear save-vs-retrieve verb, enabling a wrong pick |
Compare the field