01Safety
0.0 / 30
What changed in the harness
Selection accuracy 98→98, token cost down 0%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.2 / 30
03Economics
15.4 / 20
04Discoverability
15.7 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
5 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_recordings |
no_return_description |
List the user's most recent Clipy screen recordings, newest first. Each result carries the recording's id, title, description, duration, and status (one of ready, processing, failed, pending, queued, reflecting how ready its transcript, AI summary, and key moments are), so you can pick one to open with get_recording or get_transcript. Use this to browse when you don't have a search term; pass status to narrow to e.g. ready recordings. |
add_marker |
no_return_description |
Drop a live-timestamped narration marker into the active recording session, e.g. reproduced the bug or the fix renders correctly at mobile width. Markers become the recording's transcript chapters, so narrate as you work — that is how the recording stays agent-readable despite having no audio. A mark carries evidence in ONE of two provenances, never both: clipy-verified (assertSelector must exist; assertText must appear in that element's text and requires assertSelector; assertUrl globs the live URL — Clipy itself checks the page, so this is the strongest evidence) or driver-attested (observed + verdict, both required — you report what YOUR tooling saw; weaker than verified, falsifiable against the recorded frames). Failures are annotated into the mark as explicit FAILURES, tallied in their own segment of the recording's verification summary, and with failMode abort they discard the whole session. Pass atSeconds to backdate a mark (assertions still evaluate NOW; if that diverges from the backdated position by >2s the mark is annotated). Whenever a mark gets recorded, the call returns the mark and its outcome — the pass/fail verdict of any assertion or attestation, or a loud discard notice if failMode abort discarded the session. |
add_chapter |
no_return_description |
Drop a CHAPTER boundary into the active recording session — a mark reading === CHAPTER: <label> === at the live clock — to split the recording into named sections. Ideal for before/after recordings, e.g. a PR review: demo the base branch, add_chapter AFTER — fix applied, swap branches, demo again, then stop_recording. Chapters ride the transcript so a reader (or the summary) can see the boundaries. Returns confirmation that the chapter was inserted into the active session's transcript. |
replace_transcript |
no_return_description |
REPLACE a recording's transcript with content you author (needs the ingest scope). Use it to fix a bad speech-to-text pass, translate, or enrich a silent agent capture after upload — pass timestamped segments or a plaintext block. The summary regenerates from the new text automatically, and provenance is explicit: the transcript is marked as agent-edited, never passed off as speech-to-text. Returns confirmation that the transcript was replaced and its summary is regenerating from the new text. |
abort_recording |
no_return_description |
Discard the active recording session: closes the browser and deletes the capture; nothing is uploaded. Use this when the session captured the wrong thing or an error made it worthless. Returns confirmation that the session was aborted and its capture deleted. |
Selection evidence
Confusable tool pairs.
9 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
stop_recording |
abort_recording |
high | 'stop' and 'abort' are near-synonymous verbs with opposite outcomes (upload+save vs discard+delete); a natural task like 'stop the current session' or 'abort this recording' can land on the wrong one, either permanently deleting a wanted capture or accidentally uploading an unwanted one. |
get_agent_context |
get_context_document |
medium | Both names contain 'context' and each description claims the 'start here when handed a link/document' role, but one resolves recording ids and the other imported-video context-document ids; a user handing over a bare Clipy link without labeling it recording vs document leaves both selectable, and the wrong pick resolves nothing. |
get_context_document |
read_context_document |
medium | 'get' and 'read' are near-synonyms in user phrasing; a task like 'read the context document' or 'get its transcript' can land on get_context_document, which deliberately returns only metadata and no transcript, so the agent comes back empty and must call again. |
add_marker |
add_chapter |
medium | Both drop timeline annotations into the active recording session and add_marker's docs say markers become the transcript's chapters; a task like 'add a chapter marker at this step' or 'mark the before/after boundary' blends both namespaces, and the wrong pick mislabels a narration marker as a section boundary or vice versa. |
get_recording |
download_recording |
medium | Bare phrasing like 'here's the link, get me the recording' is genuinely ambiguous between fetching metadata (get_recording) and pulling the MP4 for local clipping/processing (download_recording); both take the same id and return different artifacts, so the wrong member gives the wrong thing. |
search_memory |
search_recordings |
medium | search_memory covers both screen recordings and imported/watched videos and is documented as the 'reach for first' tool, while search_recordings only searches titles/descriptions of personal recordings; a task like 'search my recordings for login flow' name-matches search_recordings and would silently miss context-document hits the user expected. |
list_context_documents |
get_context_document |
low | A plural task like 'get my context documents' name-matches get_context_document, but that tool requires a single id and cannot enumerate, so the agent would wrongly fail where list_context_documents (which lists) was intended. |
list_context_documents |
read_context_document |
low | A task like 'read my context documents' is ambiguous between enumerating what exists (list_context_documents) and dumping a specific one (read_context_document), which demands an id the agent doesn't have yet; a skim can pick the id-requiring tool and error out before listing. |
get_summary |
get_agent_context |
low | Both return the AI summary plus action items for the same recording id, and get_agent_context advertises itself as the one-call bundle to use first when handed a link; a 'summarize this recording' task could select the heavy bundle, burning context on frames and transcript when only the summary was requested. |
Compare the field