01Safety
0.0 / 30
What changed in the harness
Selection accuracy 98→96, token cost up 6%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.0 / 30
03Economics
19.9 / 20
04Discoverability
18.0 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
6 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
interact_page |
no_return_description |
Performs the given actions (scroll, click, hover, type, press, select, wait, navigate) in order against the page tied to sessionId, capturing them in the ongoing recording. Returns a confirmation of the actions executed. |
stop_recording |
name_restates_behavior no_return_description |
Finalizes the recording identified by sessionId and writes the captured video to a .webm file on disk. Returns the file path of the saved recording. |
convert_to_gif |
name_restates_behavior no_return_description |
Runs ffmpeg's two-pass palette generation to transcode the .webm file at webmPath into an optimized animated GIF, optionally trimming leading seconds and adjusting fps/width. Returns the file path of the generated GIF. |
convert_to_mp4 |
name_restates_behavior no_return_description |
Transcodes the .webm file at webmPath into an H.264 MP4 using the given quality (crf), for broad compatibility with social media, sharing, and embedding. Returns the file path of the generated MP4. |
record_and_gif |
no_return_description |
Opens url in a browser, records for durationSeconds, stops the recording, and converts it directly to a GIF in one call — best for simple demos. Returns the file paths of the resulting .webm and .gif files. |
list_recordings |
name_restates_behavior |
Scans directory (default ./recordings) and returns the filenames/paths of all .webm and .gif recording files found there, useful for locating outputs from prior record_page/stop_recording or convert_to_gif calls. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
convert_to_gif |
record_and_gif |
high | A task like 'make a gif of this page' could map to either calling record_and_gif directly (all-in-one) or manually record_page+stop_recording+convert_to_gif; an agent might pick convert_to_gif alone without realizing it requires a pre-existing webm, confusing which tool starts the workflow. |
record_page |
record_and_gif |
high | Both start a recording from a URL; a simple demo request ('record this page and give me a gif') is exactly the case where an agent must choose between the granular record_page (requires separate stop_recording/convert steps) and the all-in-one record_and_gif, and the tool names/descriptions overlap heavily on 'open URL and record'. |
convert_to_gif |
convert_to_mp4 |
medium | Both take a webmPath and convert format; if the user's desired output format is ambiguous (e.g. 'convert this recording so I can share it'), the agent could pick the wrong one since both are valid 'convert video' actions with similar signatures. |
record_page |
interact_page |
low | interact_page requires a sessionId from record_page so misuse as a standalone starter is unlikely, but a vague task like 'go to this page and click something while recording' could momentarily confuse which tool initiates vs performs actions. |
stop_recording |
list_recordings |
low | Both relate to finishing/retrieving recordings, but their purposes (stop a session vs list files) are distinct enough that confusion is unlikely except in a vague 'get me the recording' request. |
Compare the field