01Safety
0.0 / 30
What changed in the harness
Selection accuracy 96→98, token cost up 6%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.2 / 30
03Economics
19.9 / 20
04Discoverability
14.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
6 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
interact_page |
no_return_description |
Perform sequential actions (scroll, click, hover, type, press, select, wait, navigate) on the page being recorded for the given session ID; each action is captured in the recording. Returns once all actions have been applied. |
stop_recording |
name_restates_behavior no_return_description |
Stop the active recording for the given session ID, finalize the captured video, and save it as a .webm file. Returns the path to the saved .webm file. |
convert_to_gif |
name_restates_behavior no_return_description |
Convert a .webm video file into an optimized, web-friendly GIF using ffmpeg's two-pass palette method, optionally adjusting fps, width, and trimming the start. Returns the path to the generated .gif file. |
convert_to_mp4 |
name_restates_behavior no_return_description |
Convert a .webm video file into an MP4 (H.264) for broad compatibility with social media, sharing, and embedding, with an optional CRF quality setting. Returns the path to the generated .mp4 file. |
record_and_gif |
no_return_description |
Record the given URL for durationSeconds, then convert the result to a GIF in one call - ideal for simple demos where you want a ready-to-share animation. Returns the path to the generated .gif file. |
list_recordings |
name_restates_behavior |
List the available .webm and .gif recording files in the given directory (default: ./recordings) so you can find files to convert or share. Returns the names of the recording files found. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
convert_to_gif |
convert_to_mp4 |
low | Both accept a .webm input and convert it, so a task like 'convert this video/webm recording' without naming the target format matches either. Format-specified tasks (e.g. 'make a gif') are unambiguous because each description names its output format. |
convert_to_gif |
record_and_gif |
medium | 'Make a gif' can mean converting an existing .webm file (convert_to_gif) or recording a URL and converting it in one step (record_and_gif). The agent must infer whether the user already has a file or wants a new recording, and both tools share 'gif/convert' vocabulary. |
record_page |
interact_page |
low | A task like 'record my page interaction' is ambiguous between starting a recording (record_page) and scripting actions that get captured into video (interact_page). Descriptions otherwise diverge clearly, so confusion is only plausible for compound phrasing. |
record_page |
record_and_gif |
medium | A bare 'record this URL' task fits both: record_page opens the URL and returns a session (webm), while record_and_gif is the all-in-one that also opens and records. record_and_gif's 'best for simple demos' framing invites selection for any simple recording request, even when the user only wants raw video. |
Compare the field