01Safety
0.0 / 30
What changed in the harness
Selection accuracy 90→95, token cost up 14%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
23.6 / 30
03Economics
20.0 / 20
04Discoverability
10.2 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
10 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_containers |
no_return_description |
List Docker containers on the host, returning each container's ID, name, image, status, and ports. Set all=true to include stopped containers; by default only running containers are shown. |
container_logs |
no_return_description |
Fetch stdout/stderr log output from a Docker container, returning the log text as a string. Set id to the container ID or name, and tail to limit output to the last N lines (default 100). |
start_container |
name_restates_behavior no_return_description |
Start a stopped Docker container identified by id (container ID or name) and return its updated status once started. |
stop_container |
name_restates_behavior no_return_description |
Stop a running Docker container identified by id (container ID or name) and return its updated status once stopped. |
restart_container |
name_restates_behavior no_return_description |
Stop and then start a Docker container identified by id (container ID or name), returning its updated status once restarted. |
remove_container |
name_restates_behavior no_return_description |
Delete a Docker container identified by id (container ID or name) from the host, returning a confirmation once removed. Set force=true to remove a container that is still running. |
exec_command |
no_return_description |
Run a command inside a running Docker container identified by id, using the array in command as the executable and its arguments (e.g. ['ls', '-la']), and return the command's stdout/stderr output. |
container_stats |
no_return_description |
Query live resource usage for a running Docker container identified by id, returning CPU percentage, memory usage/limit, and network I/O statistics. |
list_images |
name_restates_behavior no_return_description |
List Docker images stored on the host, returning each image's ID, repository/tag, size, and creation date. |
remove_image |
name_restates_behavior no_return_description |
Delete a Docker image identified by id (image ID or tag) from the host, returning a confirmation once removed. Set force=true to remove an image that is in use or has multiple tags. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
remove_container |
remove_image |
medium | A task like 'remove nginx' is ambiguous about whether it targets the running container or the underlying image, since both tools share the same remove/force semantics and only differ by 'container' vs 'image' in the id. |
start_container |
restart_container |
medium | For a stopped container, a request to 'restart' it could be misrouted to start_container (docs say 'start a stopped container') instead of restart_container, since the agent may not know the container's current state and both tools accept only an id. |
container_logs |
container_stats |
low | A vague request like 'check on container X' could plausibly be served by either logs or stats, though the descriptions ('logs' vs 'CPU/memory/network stats') are distinct enough to usually disambiguate. |
Compare the field