01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100%, destructive-action safety rate 0% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.9 / 30
03Economics
14.9 / 20
04Discoverability
8.9 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
7 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
retroarch_read_memory |
retroarch_read_ram |
high | Both read emulated memory and share nearly identical descriptions, differing only in address space (libretro core map vs CHEEVOS). A task like 'read the RAM at 0xFF0000' matches the tool named read_ram by literal name, but the docs say read_memory is the preferred tool; an agent choosing by name alone picks the wrong address space and gets wrong data. |
retroarch_write_memory |
retroarch_write_ram |
medium | Same address-space trap as the read pair: 'write to RAM' name-matches retroarch_write_ram, while retroarch_write_memory is documented as preferred. Both accept address+bytes and are marked destructive, so an agent picks the wrong one when the user just says 'poke memory value X at Y' without specifying the space. |
retroarch_load_state_current |
retroarch_load_state_slot |
medium | Both load savestates with overlapping descriptions. A user saying 'load my save state' or 'load the state from a slot' is ambiguous about whether the current-slot pointer should be used/left alone; the tools differ on whether they touch the slot pointer, and the generic task does not disambiguate. |
retroarch_load_state_slot |
retroarch_state_slot_plus |
medium | A task like 'go to the next save slot' or 'switch to save slot 3' could be satisfied by walking the pointer (state_slot_plus) or by loading from a numbered slot (load_state_slot); the descriptions overlap heavily on slot/pointer/current vocabulary, so an agent may select the wrong one and either mutate the pointer or unexpectedly overwrite live state. |
retroarch_load_state_slot |
retroarch_state_slot_minus |
medium | Mirror of the plus case: 'go back to the previous save slot' is ambiguous between decrementing the current-slot pointer and loading a specific slot's state. Both tools mention slots, current pointer, and loaded state with near-identical wording, making mistaken selection plausible. |
retroarch_state_slot_plus |
retroarch_state_slot_minus |
medium | These differ only in direction and share every description token except plus/minus. A task phrased without an explicit direction — 'change the save slot', 'cycle to another slot', 'bump the selected slot' — gives the agent no basis to choose, and either selection may be wrong depending on intended direction. |
retroarch_get_status |
retroarch_get_config |
medium | Both are parameterless 'get' queries whose descriptions reference each other and share config/status/state vocabulary. A vague task like 'check the current state/configuration of the emulator' or 'what are the current settings' does not clearly map to run-state versus static config, so an agent can return the wrong kind of information. |
Compare the field