01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 7%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.1 / 30
03Economics
19.5 / 20
04Discoverability
11.6 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
barevalue_api_status |
no_return_description |
Check Barevalue API health, queue depth, and estimated processing times. Returns the current health status, queue depth, and estimated processing times, so you can verify the service is operational before submitting orders. No authentication required. |
barevalue_validate |
no_return_description |
Pre-check an audio file from a public URL before submission. Validates speech content (requires minimum 10% speech) and detects music-only content; does NOT charge credits. Returns the validation result indicating whether the file has sufficient speech content and is ready for submission. Use for external URLs before submitting; files uploaded via barevalue_upload can skip validation and go directly to submit. |
barevalue_submit |
params_unexplained no_return_description |
Submit an uploaded file (using the order_id and s3_key returned by barevalue_upload) for editing. editing_mode='ai' (default) performs AI podcast editing of audio only (up to 60 min), billed against credits/subscription minutes. editing_mode='human' routes to a professional human editor for audio or video, billed at standard per-minute rates (credits first, then the default card on file for the remainder); use it for white-label/human-only accounts and for video. podcast_name and episode_name are required to identify and label the episode in the transcript and output; episode_number, special_instructions, host_names, and guest_names are optional labels/customization. Returns the submitted order (e.g., order_id) which can be tracked via barevalue_status. |
barevalue_submit_url |
params_unexplained no_return_description |
Submit a podcast for AI editing using a public URL; the file will be downloaded and processed. Useful for files hosted on Dropbox, Google Drive (with a public link), or other file hosting services. podcast_name and episode_name are required to identify and label the episode; episode_number, special_instructions, host_names, and guest_names are optional labels/customization. Returns the submitted order (e.g., order_id) which can be tracked via barevalue_status. |
barevalue_list_orders |
no_return_description |
List recent orders with their status. Useful for checking on multiple orders or finding a specific order. Returns a paginated list of orders (with their current status), controlled by page, per_page, and an optional status filter. |
barevalue_webhooks_list |
no_return_description |
List all configured webhooks for your account. Webhooks send notifications when orders complete, fail, or are refunded. Returns each webhook's ID, URL, subscribed events, and active status. |
barevalue_webhook_update |
no_return_description |
Update an existing webhook's URL, events, or active status, identified by webhook_id. Returns the updated webhook configuration. |
barevalue_webhook_delete |
no_return_description |
Permanently delete a webhook identified by webhook_id. This cannot be undone. Returns confirmation that the webhook was deleted. |
Selection evidence
Confusable tool pairs.
8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
barevalue_api_status |
barevalue_status |
medium | A vague task like 'check the status' is ambiguous: api_status reports API/service health while status reports a specific order's state. Without specifying service vs order, an agent could pick the wrong one. |
barevalue_submit |
barevalue_submit_url |
medium | Both are 'submit for editing' tools. A task like 'submit my podcast for editing' doesn't reveal whether the file is already uploaded (needs order_id/s3_key via submit) or hosted at a URL (submit_url), so the agent can pick the wrong path and fail. |
barevalue_upload |
barevalue_submit |
medium | Local-file workflow requires upload then submit. A task like 'please edit this audio file for me' could make the agent call submit directly (which needs order_id/s3_key it doesn't have) or only upload and stop instead of completing the order. |
barevalue_webhook_create |
barevalue_webhook_update |
low | Vague phrasing like 'set up a webhook at this URL' or 'change my webhook config' could lead the agent to create a new webhook when one should be updated, or update an existing one when a new one was meant. |
barevalue_webhook_update |
barevalue_webhook_rotate_secret |
low | A task 'update my webhook's signing secret' names the generic 'update' verb, which could select barevalue_webhook_update, even though only rotate_secret generates a new secret and update has no secret parameter. |
barevalue_webhook_create |
barevalue_webhook_rotate_secret |
low | A task like 'I need a new secret for my webhook' might trigger create (making a whole new webhook) rather than rotate_secret on the existing webhook, since both can sound like 'make something new'. |
barevalue_api_status |
barevalue_validate |
low | Both describe 'checking' something: a terse task such as 'check this' or 'check the status of this audio' could route to api_status (service health) instead of validate (audio speech-content check) or vice versa. |
barevalue_validate |
barevalue_submit_url |
low | Both take a file_url and concern published URLs. 'Process/check this podcast from this Dropbox link' is ambiguous between validating the file first (barevalue_validate) and directly submitting it (barevalue_submit_url), especially when the agent should do both. |
Compare the field