01Safety
0.0 / 30
What changed in the harness
Selection accuracy 83→80, token cost up 3%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.7 / 30
03Economics
20.0 / 20
04Discoverability
14.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
klaket_job_status |
no_return_description |
Check the status of a Klaket ingest job. Returns the job's current status as one of 'queued', 'processing', 'done', or 'failed'; poll this until it returns 'done' for the id from klaket_ingest. |
klaket_find_moment |
no_return_description |
Search a processed video for moments matching a query. Returns the matching moments as timestamped transcript lines, on-screen text, or scene descriptions with their timestamps. The job must be 'done' before calling this. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
klaket_get_result |
klaket_find_moment |
medium | Both act on a completed job and expose the same kind of data (timestamped transcript, scene descriptions, on-screen text). Wording like 'get the moments about the pricing slide' or 'find the processed content for this video' could plausibly route to the wrong one since one returns the full markdown while the other runs a search. |
klaket_job_status |
klaket_get_result |
medium | A task such as 'what's the result of my job?' is genuinely ambiguous: 'result' matches klaket_get_result's name, but asking about a job's outcome maps to klaket_job_status. The two are easy to conflate for vague status-vs-content requests. |
klaket_ingest |
klaket_get_result |
low | Asking to 'get the result for this video/URL' before processing has started could pick klaket_get_result on name match alone, when klaket_ingest must be called first. Low plausibility because descriptions state the required pipeline order. |
Compare the field