01Safety
0.0 / 30
What changed in the harness
Selection accuracy 97%, destructive-action safety rate 0% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.6 / 30
03Economics
15.8 / 20
04Discoverability
8.1 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
9 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
pine_read16 |
pine_read32 |
medium | A task like 'read the value at 0x00200000' omits bit-width; both tools are near-identical pure reads differing only in width, and domain hints overlap (HP/score vs timestamps/counters), so the agent can silently pick the wrong sized read and get a truncated or wrong value. |
pine_read16 |
pine_read64 |
medium | Width-less reads ('read the value at X') are ambiguous between 16 and 64-bit; both descriptions give no default, and a wrong-width read returns a corrupted value with no error, so an agent guessing width can pick the wrong member. |
pine_read32 |
pine_read64 |
medium | Both target counters/timestamps/pointers ('32-bit is the lower half of 64-bit pointers'), so 'read the counter/pointer at X' is genuinely ambiguous; the alignment and return-type differences (number vs decimal string) don't help disambiguate a width-less task. |
pine_read8 |
pine_read32 |
medium | read8 lists 'counters' and read32 lists 'large counters'/timestamps, so 'read the counter/flag at X' or any bare-address read gives no width signal, allowing the agent to pick byte vs word incorrectly. |
pine_write16 |
pine_write32 |
medium | 'Set the value at 0x... to 5000' carries no bit-width; the write tools differ only in width/range and are both destructive, so a width-less cheat/poke task can land on the wrong member and overwrite too few or too many bytes. |
pine_write16 |
pine_write64 |
low | Generic 'write value at X' tasks usually signal 64-bit poorly, but a task mentioning a 'big ID/pointer' could be written at 16-bit; confusion is possible but 64 is an outlier pick, so probability is low. |
pine_write8 |
pine_write64 |
low | Only confused for width-less 'poke' tasks where the agent must guess; users usually say 'byte' (write8) or specify a 64-bit value, so selecting the extreme-width member wrongly is possible but uncommon. |
pine_ping |
pine_get_status |
medium | 'Is the emulator running?' is ambiguous: pine_ping verifies reachability/liveness ('OK — emulator: version'), while pine_get_status reports run state (running/paused/shutdown); an agent could pick either for a liveness-style question and miss the other's semantics. |
pine_get_info |
pine_get_status |
low | Both return the Status field (get_info bundles Status with metadata), so 'what's the emulator's status/state' could select either; however both descriptions explicitly cross-reference the other and get_status is clearly the cheap 1-call choice, keeping real confusion low. |
Compare the field