01Safety
0.0 / 30
What changed in the harness
Selection accuracy 95%, destructive-action safety rate 0% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.2 / 30
03Economics
18.3 / 20
04Discoverability
14.5 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
kb_repair |
kb_repair_aliases |
high | The names differ by one token and both follow the same list-a-worklist-then-fix-via-kb_write flow. Worse, kb_repair's description explicitly says to run it 'when the user asks to repair, fix, or clean up the notebook', so a generic task like 'repair the notebook' or 'fix the broken/wrong aliases' can route to the wrong member unless the agent absorbs the missing-vs-stale cross-reference buried in kb_repair_aliases. |
kb_search |
kb_lookup |
medium | kb_lookup's name invites concept lookup, but its definition is strictly address-keyed and requires an exact repo path. A task phrased as a concept or symptom — 'look up what the notebook knows about the logout loop' or 'about symbol X' — without a full path plausibly triggers kb_lookup (possibly with a guessed path) instead of the fuzzy, concept-keyed kb_search that explicitly accepts symptoms, symbols, and file names. |
kb_lookup |
kb_status |
low | Both answer 'what does the notebook know at this path': kb_lookup exhaustively for one exact file, kb_status(paths) returns the notes anchored at each path with freshness. A task like 'check whether anything is known about these files before I edit' can plausibly route to either, since the two surfaces overlap on per-path anchored notes. |
kb_status |
kb_repair |
low | Both are notebook-health surfaces: kb_status reports stale/superseded counts and kb_repair lists unfindable notes. A task like 'how many notes are broken' or 'what in the notebook needs fixing' reads 'stale' and 'unfindable' as the same 'needs repair' category, so an agent could select the wrong one. |
Compare the field