01Safety
30.0 / 30
What changed in the harness
Selection accuracy 93→93, token cost up 2%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.3 / 30
03Economics
19.9 / 20
04Discoverability
11.2 / 20
Highest-impact fix
Estimated gain +9 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_unread_conversations |
no_return_description |
List only guest message threads that have unread messages, scanning up to the given limit conversations. Returns a compact Markdown table of the unread conversations (conversation id, guest, linked listing/reservation, unread state, and last activity time), up to the limit, so you can see which guests are waiting on a reply. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
list_conversations |
list_unread_conversations |
high | Both list guest message threads with nearly identical descriptions; a task like 'show me my guest conversations' or 'tell me what messages I have' matches either, and only the unread tool hints it answers 'any guests waiting on a reply'. |
list_reservations |
get_reservation |
medium | A task like 'get my reservations' or 'pull up the reservation for guest X' (without an id) both match the word 'reservation'; an agent may pick get_reservation and fail on the required id instead of listing first. |
list_unread_conversations |
get_conversation_messages |
medium | A task like 'show me the messages I've received' or 'get the conversation content' overlaps both: one lists threads with unread state, the other fetches full message history by id, and neither alone clearly covers finding-and-reading. |
get_listing |
get_reservation |
low | Both are 'fetch full details by id' tools; a vague task like 'get the full details for id 7' is ambiguous, though well-specified tasks name listing vs reservation explicitly. |
Compare the field