01Safety
0.0 / 30
What changed in the harness
Selection accuracy 86→88, token cost up 2%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.7 / 30
03Economics
19.9 / 20
04Discoverability
14.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_conversions |
no_return_description |
List all supported source-to-target conversion routes. Optionally pass a source_format (e.g. "docx") to see only conversions FROM that format. Returns the available conversion routes, i.e. the supported source formats and their corresponding target formats. |
do_file_job |
no_return_description |
Describe what you want done to a file in plain language (e.g. "translate this contract to German", "pull every table out of this PDF into Excel", "shrink this video to under 25MB", "convert this to PDF"). The instruction is routed to the right job automatically and runs immediately; if the request is not supported yet you get an honest explanation of what is. Provide EITHER source_url OR base64_content, plus a filename with extension. Returns a job_id that you can pass to check_job to monitor status and retrieve preview, checkout, or download links. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
convert_file |
compress_file |
medium | Natural phrasing like 'convert this video to under 10MB' or 'convert this file to a smaller size' mixes the verb 'convert' with a size target; convert_file handles only format changes with no size control while compress_file is size-targeted, so the agent may select the wrong one. |
convert_file |
translate_file |
low | Tasks phrased as 'convert this document to Spanish/German' use 'convert' but specify a language rather than a format extension; the agent could pick convert_file even though translate_file is intended. |
convert_file |
do_file_job |
medium | do_file_job's description explicitly lists 'convert this to PDF' as a supported plain-language example, so a task like 'convert this DOCX to PDF' could plausibly be routed to the generic router instead of the dedicated convert_file. |
translate_file |
do_file_job |
medium | do_file_job advertises 'translate this contract to German' as an example, so a task such as 'translate this report to French' may be sent to the generic job tool rather than the dedicated translate_file. |
compress_file |
do_file_job |
medium | do_file_job advertises 'shrink this video to under 25MB', so a task like 'compress this PDF under 5MB' could legitimately be routed to the generic router instead of the dedicated compress_file. |
Compare the field