01Safety
30.0 / 30
What changed in the harness
Selection accuracy 98→100, token cost up 20%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.6 / 30
03Economics
20.0 / 20
04Discoverability
12.9 / 20
Highest-impact fix
Estimated gain +7 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
codewiki_search_repos |
name_restates_behavior params_unexplained no_return_description |
Find repositories indexed by codewiki.google by keyword. Use after a codebase or topic is known, to locate matching repos before fetching their wiki content. Pass `query` to specify the repository name/topic to search for; optionally set `limit` to cap the number of results (default 10, max 50). Returns a list of matching repository identifiers and basic metadata, usable with codewiki_fetch_repo. |
codewiki_fetch_repo |
name_restates_behavior params_unexplained no_return_description |
Retrieve the generated wiki documentation for a repository indexed by codewiki.google. Provide `repo` to identify the target repository; set `mode` to 'aggregate' (default) to return the combined wiki content for the whole repo, or 'pages' to return the wiki split by page. Returns the requested wiki content for the given repository. |
codewiki_ask_repo |
name_restates_behavior params_unexplained no_return_description |
Answer a natural-language question about a repository indexed in codewiki.google, grounded in its generated wiki content. Pass `repo` to target the repository and `question` to state the query; optionally supply `history` (up to 20 prior user/assistant turns, each with `role` and `content`) to provide conversational context. Returns the answer along with any supporting wiki context for the question. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
codewiki_fetch_repo |
codewiki_ask_repo |
medium | Both operate on a single known repo. A task like 'get me information about repo X' or 'tell me about this repository' could plausibly map to either: fetch pulls the generated wiki content, while ask answers a natural-language question about the repo. Without a clear 'question' phrasing, an agent may pick the wrong one. |
codewiki_search_repos |
codewiki_ask_repo |
medium | The shared 'search' vocabulary is ambiguous. A task like 'search this repository for how it does auth' sounds like a search but actually targets one repo with a natural-language question, so an agent may call search_repos (across-repo search) instead of ask_repo, which requires a repo+question. |
codewiki_search_repos |
codewiki_fetch_repo |
low | Search discovers repos by query while fetch retrieves wiki content for a known repo. A task like 'look up the wiki for the kubernetes repo' or 'find me the docs for project X' could, in edge cases, route to search_repos when fetch_repo (which needs an exact repo name) is intended, but the required-input difference makes real confusion unlikely. |
Compare the field