01Safety
0.0 / 30
What changed in the harness
Selection accuracy 98→98, token cost up 20%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
24.4 / 30
03Economics
20.0 / 20
04Discoverability
11.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
7 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_buckets |
name_restates_behavior no_return_description |
Retrieves every S3 bucket owned by the authenticated AWS account, with no filtering options. Returns an array of bucket names along with their creation dates. |
list_objects |
name_restates_behavior params_unexplained no_return_description |
Lists keys stored in a given S3 bucket, optionally narrowed to those starting with a prefix and capped by a max count. 'bucket' is the name of the S3 bucket to query. Returns an array of matching object entries (key, size, last-modified timestamp) up to maxKeys results. |
get_object |
params_unexplained no_return_description |
Downloads and reads the contents of an S3 object as text. 'bucket' is the name of the S3 bucket containing the object, and 'key' is the object's full path/identifier within that bucket. Returns the object's body decoded as a UTF-8 text string. |
put_object |
params_unexplained no_return_description |
Uploads text content to an S3 object, creating it or overwriting it if it already exists. 'bucket' is the destination S3 bucket name, and 'key' is the path/identifier to store the object under. Returns confirmation of the upload including the object's key and ETag. |
delete_object |
name_restates_behavior params_unexplained no_return_description |
Permanently removes a single object from an S3 bucket by its key; this cannot be undone unless bucket versioning is enabled. 'bucket' is the S3 bucket name, and 'key' is the object's path/identifier to delete. Returns confirmation that the delete request succeeded. |
presigned_url |
name_restates_behavior params_unexplained no_return_description |
Creates a time-limited, signed URL that grants temporary access to a private S3 object without requiring AWS credentials. 'bucket' is the S3 bucket name, 'key' is the object's path/identifier, and 'expiresIn' controls how many seconds the URL remains valid (default 3600). Returns the generated presigned URL string. |
bucket_info |
params_unexplained no_return_description |
Checks if a bucket exists and get basic info. 'bucket' is the name of the S3 bucket to check. Returns whether the bucket exists along with metadata such as region and creation date when available. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
list_buckets |
list_objects |
medium | A vague request like 'list what's in my S3' or 'show my S3 storage' doesn't specify whether the user wants bucket names or the contents of a particular bucket, and an agent might pick list_buckets when list_objects (with a bucket) was intended, or vice versa if a bucket name is mentioned ambiguously. |
list_buckets |
bucket_info |
medium | Requests like 'check my S3 bucket' or 'get bucket details' could plausibly route to either list_buckets (to enumerate/find it) or bucket_info (to check existence/info for a named bucket), especially when the user references 'the bucket' generically without being clear if they want a full listing or a single lookup. |
list_objects |
get_object |
medium | A phrase like 'get the objects in this bucket' or 'get me the files in uploads/' is ambiguous between listing object keys (list_objects) and fetching the content of a specific object (get_object), particularly if no explicit key is given. |
get_object |
put_object |
low | Both operate on object content and share 'text' framing; a task phrased as 'update the object with new content' could momentarily suggest reading (get) is needed before the intended write (put), but the verbs get/put are otherwise clearly opposite, keeping confusion unlikely. |
Compare the field