01Safety
0.0 / 30
What changed in the harness
Selection accuracy 98→93, token cost down 2%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.9 / 30
03Economics
15.3 / 20
04Discoverability
15.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
5 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_recordings |
no_return_description |
List the user's most recent Clipy screen recordings, newest first, optionally filtered by processing status. Use this to browse when you don't have a search term. Returns an array of recording summaries (id, title, description, duration, and processing/transcript/summary status) for each match. |
add_marker |
no_return_description |
Drop a live-timestamped narration marker into the active recording session ('reproduced the bug', 'the fix renders correctly at mobile width'). Markers become the recording's transcript chapters, so narrate as you work — they are how the recording stays agent-readable despite having no audio. A mark can carry evidence in ONE of two provenances, never both: (1) CLIPY-VERIFIED via assertSelector/assertText/assertUrl, checked by Clipy itself against the recorded page — the strongest evidence; or (2) DRIVER-ATTESTED via observed+verdict, where you report what your own tooling saw — weaker than clipy-verified but stronger than plain prose. Marks default to the live recording clock; pass atSeconds to backdate one. Failures are annotated as explicit FAILURES and tallied in the recording's verification summary; failMode 'abort' discards the whole session. Returns the created marker's text, resolved timestamp, and (if evidence was supplied) its pass/fail verdict. |
add_chapter |
no_return_description |
Drop a CHAPTER boundary into the active recording session — a mark reading '=== CHAPTER: <label> ===' at the live clock — to split a recording into named sections, e.g. before/after comparisons. Chapters ride the transcript so a reader or the AI summary can see the boundaries. Returns the created chapter marker with its label and timestamp. |
replace_transcript |
no_return_description |
REPLACE a recording's transcript with content you author (needs the 'ingest' scope) — use it to fix a bad speech-to-text pass, translate, or enrich a silent agent capture after upload. The summary regenerates from the new text automatically, and the transcript is marked as agent-edited, never passed off as speech-to-text. Returns the updated transcript record, including its stored segments and regeneration status. |
abort_recording |
no_return_description |
Discard the active recording session: closes the browser and deletes the capture without uploading anything. Use this when the session captured the wrong thing or an error made it worthless. Returns a confirmation that the session was discarded. |
Selection evidence
Confusable tool pairs.
10 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_context_document |
read_context_document |
high | Both take the same 'id' param and operate on the same context document; a task like 'get me the info on this video document' is ambiguous between metadata-only (get_context_document) and full transcript content (read_context_document) — an agent could easily call the wrong one first. |
get_agent_context |
get_context_document |
medium | Names are nearly identical ('agent context' vs 'context document') but operate on entirely different libraries — recordings vs imported videos — and take an id from different id spaces; an agent handed a generic 'get context for this' request could pick the wrong tool, though the descriptions do clarify domain if read carefully. |
list_context_documents |
get_context_document |
low | Different verbs (list vs get) and different required params (get needs id) make selection fairly clear, but a vague task like 'find info about this imported video' could momentarily cause confusion about which to call first since both mention document metadata. |
list_context_documents |
read_context_document |
low | Distinct verbs (list/read) and read requires an id, so confusion is unlikely except in a vague 'show me the context document' request where an agent might not realize it needs to list first to get the id. |
get_recording |
get_summary |
medium | A task like 'get me the summary of this recording' could lead an agent to call get_recording (which only reports summary status, not content) instead of get_summary which returns the actual TL;DR/key points. |
get_recording |
get_transcript |
medium | Similarly, 'get the recording' could be interpreted as wanting the transcript content rather than metadata; get_recording only returns transcript status, not the transcript itself, so an agent might pick the wrong one for 'what was said in this recording'. |
get_transcript |
get_summary |
low | Both fetch recording content by id and could be conflated in a vague request like 'tell me what's in this recording', though 'transcript' vs 'summary' wording is usually distinct enough to avoid real confusion. |
stop_recording |
abort_recording |
high | Both end the active recording session with no params, but stop_recording uploads while abort_recording discards; a task like 'end the recording' or 'cancel the recording session' is genuinely ambiguous about which semantics are wanted, and picking wrong is destructive/irreversible. |
search_memory |
search_recordings |
medium | Both search by keyword/topic across recordings; a task like 'find where I talked about the login bug' could go to either, but search_recordings only covers screen recordings (title/description) while search_memory covers both recordings and context docs with semantic matching on spoken content — an agent might miss the broader/more precise tool. |
get_recording |
download_recording |
low | Both take a recording id, but 'download' vs 'get' verbs are distinct enough; low risk of confusion except in a loosely worded 'grab this recording for me' request. |
Compare the field