01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→98, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.5 / 30
03Economics
19.0 / 20
04Discoverability
13.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
roamzy_status |
no_return_description |
Check Roamzy API operational status. Returns a status object including a `purchases_paused` boolean flag (and related service-state fields). Agents MUST call this before purchase attempts and back off if `purchases_paused=true`. |
roamzy_me |
no_return_description |
Get the current Roamzy account tied to this session. Returns account details such as the account identifier, USDT balance, and whether a Google/Telegram identity is linked. ⚠️ This MCP being connected does NOT mean the user already has a Roamzy account: in anonymous mode (no ROAMZY_API_TOKEN env), the FIRST authed call — including this one — auto-mints a fresh anonymous account. Don't tell the user «you're already a Roamzy customer» based on MCP presence; wait until after roamzy_me or roamzy_create_order returns successfully. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
roamzy_status |
roamzy_order_status |
medium | A vague 'check the status' request from the user is ambiguous between checking overall API/pause status (roamzy_status) and checking a pending purchase's progress (roamzy_order_status); an agent without careful reading of the shared 'order_id' requirement could pick either. |
roamzy_list_esims |
roamzy_get_esim |
medium | A request like 'show me my eSIM balance/status' could be served by list_esims (which already returns status+balance for all eSIMs) or mistakenly routed to get_esim, which requires an internal esim_id the agent doesn't yet have — risking a premature/invalid call instead of listing first. |
roamzy_get_esim |
roamzy_order_status |
medium | 'Is my eSIM ready yet?' spans both tools depending on purchase stage — order_status tracks payment confirmation while get_esim returns the activation QR/lpa_url; an agent unsure whether the order finished could call the wrong one. |
roamzy_list_countries |
roamzy_country_detail |
low | General rate questions ('what are Roamzy's rates?') vs destination-specific ones ('rate for Japan?') are usually distinguishable by whether a country is named, but a task like 'tell me about rates' without a clear country could plausibly trigger either tool. |
roamzy_me |
roamzy_referral |
low | Both are parameterless 'get my account info' style calls; a vague request like 'show my Roamzy account details' could be misrouted to referral (earnings/referral link) instead of account info, or vice versa. |
Compare the field