01Safety
0.0 / 30
What changed in the harness
Selection accuracy 98→100, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.1 / 30
03Economics
19.8 / 20
04Discoverability
14.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
4 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
authenticate |
no_return_description |
Connects the user's Spotify account via OAuth. Opens a browser on this machine for approval and stores tokens locally. Returns whether authentication succeeded and which account is now connected. Use when other tools report you are not authenticated, or to switch accounts. |
playback |
no_return_description |
Applies a transport control to the active playback: pause, resume, next, previous, seek (value = seconds), volume (value = 0-100), shuffle (value = on/off), repeat (value = off/context/track). Returns a confirmation of the applied action and the resulting playback state. |
modify_playlist |
no_return_description |
action=create makes a new playlist (private by default); action=add_items appends track URIs to an existing playlist. Does not delete or remove anything. Returns the new playlist's id/URI for create and a confirmation that the tracks were added for add_items. |
library |
no_return_description |
Saves, removes, or checks items in the user's Spotify library ('liked'). Works with any content URI: tracks, albums, artists (follow), shows, episodes. Returns the result of the operation, including whether each URI is saved for check. To save the current song, get its URI from now_playing first. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_playlists |
get_playlist_items |
medium | Both are read tools whose names differ only by plural 'playlists' vs 'items', and both accept playlist references. A task phrased like 'show me what's in my playlists' or 'get my playlist contents' could plausibly route to either tool: get_playlists returns playlists with item counts while get_playlist_items returns the actual tracks, so the 'what's in my playlists' phrasing is genuinely under-specified. |
get_playlists |
modify_playlist |
low | One tool reads/retrieves playlists while the other creates or adds items to a playlist. The action-verb distinction ('list/'show' vs 'create'/'add') is clear in natural language and the tool descriptions make the read/write split obvious, so selection confusion is unlikely. |
get_playlist_items |
modify_playlist |
low | get_playlist_items lists the tracks in a playlist whereas modify_playlist only writes (create or add_items). Tasks like 'what tracks are in this playlist' versus 'add a track' are semantically distinct, and only modify_playlist mentions creating or appending, so mis-selection is improbable. |
Compare the field