01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100%, destructive-action safety rate 100% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.6 / 30
03Economics
19.3 / 20
04Discoverability
18.8 / 20
Highest-impact fix
Estimated gain +1 pointDisambiguate overlapping tool choices
Give confusable tools explicit decision boundaries and examples of when to choose one instead of the other.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
10 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
recent_work |
commit_context |
medium | Both return per-commit subject/data/file lists for repositories. A task like "summarize the commits from [author] last week" could misfire to commit_context (which takes a sha/revision, not an author), or a task like "show me details of commit abc123" could misfire to recent_work (which filters by author/time, not SHA). A natural-language mention of 'commits' with a target name, date range, or SHA can be read either way since the agent must infer whether it is a person/time query or a single-commit query. |
introducing_pr |
commit_context |
medium | Both take a commit SHA and describe the linked PR, noting GitHub API fallback. A task like "which PR does this commit belong to" or "what PR introduced this change" could route to commit_context (which already parses/reports the linked PR) instead of introducing_pr, or route to introducing_pr when full commit-level context is wanted. With a SHA input, either tool superficially satisfies the request, so selection depends on fine-grained reading of the descriptions. |
introducing_pr |
branch_hygiene |
low | A task phrased as 'which PR introduced this line/branch' could theoretically touch both, but branch_hygiene only inventories branches with ahead/behind/stale data and never resolves lines or PRs, while introducing_pr explicitly finds the introducing PR for a line/commit. The descriptions disambiguate cleanly; realistic natural-language tasks do not straddle both intents. |
branch_hygiene |
commit_context |
low | branch_hygiene is branch-inventory (merged status, staleness, ahead/behind) while commit_context is a single-commit lookup keyed by SHA. Even a request mentioning 'last commit date' or 'author' on a branch points to branch_hygiene's branch list; commit_context requires an explicit SHA. The descriptions do not create believable overlap. |
branch_hygiene |
recent_work |
low | branch_hygiene lists branches with last-commit metadata; recent_work lists a single author's commits in a time window. A request such as 'what has the team committed recently' is clearly recent_work (author/time scope), and 'find unmerged branches' is clearly branch_hygiene. Distinct scopes and explicit staleness/merged semantics prevent real confusion. |
co_change |
branch_hygiene |
low | co_change requires a `file` input and answers 'files that change together with X'; branch_hygiene returns branch statistics with no file input. The required-file vs branch-only scopes are disjoint enough that a natural-language task does not plausibly map to the wrong tool. |
co_change |
commit_context |
low | co_change is file-to-file coupling (requires `file`, mines many commits), commit_context is single-commit lookup (requires `sha`). Overlap is nil: the natural-language targets ('what else changes with this file' vs 'what did this commit change') are clearly tied to distinct required parameters. |
co_change |
recent_work |
low | Both touch commit/file mining but with opposite entry points: co_change keys on a single `file` to list co-changing files; recent_work keys on an `author`/time window to list that author's commits. Tasks like 'what touched X' vs 'what did [author] touch' each select one tool unambiguously; the descriptions distinguish the parameters explicitly. |
introducing_pr |
co_change |
low | introducing_pr traces a line/commit to its introducing PR; co_change mines files that historically change with an input file. A task like 'what PR changed this code' is plainly introducing_pr and 'what else is affected if I edit X' is plainly co_change. No plausible cross-route in natural language. |
introducing_pr |
recent_work |
low | introducing_pr resolves a line/commit to its introducing PR; recent_work lists an author's commits in a time window. The descriptions are directionally opposite (trace-to-PR vs list-recent-author-work), leaving no realistic task that points at both. |
Compare the field