01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost down 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.8 / 30
03Economics
19.2 / 20
04Discoverability
13.9 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
impri_report_result |
no_return_description |
Report whether you successfully executed an approved action. Closes the audit loop so the operator sees "executed" or "execute_failed" in the inbox alongside the original action and decision — always call this after attempting an approved action, even on failure. Statuses: "executed" (action carried out successfully) or "execute_failed" (execution attempt failed; include the error in detail). Returns { action_id, status } confirming the recorded outcome. |
impri_create_watcher |
params_unexplained |
Create a watcher that monitors external sources (RSS feeds, Reddit, URL diffs) and delivers matching items to the approval inbox or a webhook. The watcher runs on the schedule you specify, deduplicates items by URL/content-hash, and delivers only new matches; the first run establishes a baseline and does not generate alerts. The spec object defines the watcher: name is the display label; kind selects the source type (e.g. "rss", "reddit"); config holds kind-specific connection details such as a feed url; keywords/keywords_none include or exclude items by matching text against them; min_score sets the minimum keyword-match score required before an item alerts; schedule.every sets the run interval and schedule.jitter adds a randomized delay to spread load. Returns { watcher_id, name, kind, status, next_run_at }. |
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
impri_create_watcher |
impri_create_watcher_from_preset |
high | A vague request like 'set up a watcher for GitHub releases' can be satisfied by either tool — the agent might hand-build a spec via impri_create_watcher instead of discovering and using the simpler preset-based impri_create_watcher_from_preset, since both return the same watcher_id/name/kind/status shape and neither name signals which is the 'default' path. |
impri_list_watchers |
impri_list_watcher_presets |
medium | A phrasing like 'what watchers are available' or 'show me the watchers' is ambiguous between listing already-configured watchers (impri_list_watchers) and listing available preset templates to create one from (impri_list_watcher_presets); both share 'list' and 'watcher' tokens and only differ by the easily-missed 'presets' suffix. |
Compare the field