01Safety
0.0 / 30
What changed in the harness
Selection accuracy 96%, destructive-action safety rate 0% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.3 / 30
03Economics
13.8 / 20
04Discoverability
7.9 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
7 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
bizhawk_press_buttons |
bizhawk_press_buttons_multi |
medium | Both describe the same joypad/button semantics and share a near-identical vocabulary (buttons, player, frame, joypad.set, released, core-specific button names). A task like 'make player 2 press Right this frame' could pull bizhawk_press_buttons because it exposes a `player` parameter, when simultaneous multi-controller input requires bizhawk_press_buttons_multi; the one-frame-per-call simultaneity constraint is subtle and easy to miss. |
bizhawk_read16 |
bizhawk_read32 |
low | Unspecified width tasks like 'read the value at 0x100' fit both: the descriptions are mirror-images (same address/domain/endian parameterization) and differ only in bit-width guidance (16-bit for HP/score/coords vs 32-bit for pointers/counters). The agent must infer field size from context, so a wrong-width pick is plausible. |
bizhawk_read8 |
bizhawk_read16 |
low | A task like 'read the HP at 0x200' without stating the field width is genuinely ambiguous: both accept the same address/domain shape and both are described as pure unsigned reads; whether a game value is a u8 or u16 is domain knowledge, not recoverable from the tool names alone. |
bizhawk_read8 |
bizhawk_read32 |
low | Same width-inference ambiguity: 'read the counter at that address' matches both signatures, and the natural-language term 'value' maps equally well to an 8-bit byte and a 32-bit dword; the agent can pick either from the interchanged-size descriptions. |
bizhawk_write16 |
bizhawk_write32 |
low | Both are destructive little-endian writes sharing identical address/value/domain schema; a task like 'set the timer at 0x100 to 700' gives no bit-width, so the agent must guess whether the field is 16- or 32-bit, making a wrong-width poke plausible. |
bizhawk_write8 |
bizhawk_write32 |
low | Same underlying ambiguity as the other width pairs: 'write 42 to 0x100' and 'poke the value at that address' both match write8 and write32 (each accepts address+value+domain); size is a semantic judgment the descriptions only weakly steer. |
bizhawk_write8 |
bizhawk_write_range |
low | A task like 'write the byte 0x42 at 0x100' could route to either: write_range explicitly accepts a single-element byte array and both overwrite memory at an address, so a yes-no read of the descriptions leaves the single-byte-vs-range choice mildly ambiguous. |
Compare the field