01Safety
0.0 / 30
What changed in the harness
Selection accuracy 95→97, token cost up 9%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.9 / 30
03Economics
17.4 / 20
04Discoverability
15.5 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
9 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
ask-codex |
no_return_description |
Execute Codex CLI with file analysis (@syntax), skills ($syntax), model selection, and safety controls. Supports changeMode for structured OLD/NEW edits. Returns the Codex response, optionally including a reasoning/thinking section and configuration metadata. |
review-changes |
no_return_description |
Run a code review against the current repository using the Codex CLI native review subcommand, targeting uncommitted changes, a base branch, or a specific commit. Returns the review findings for the requested scope. |
ping |
no_return_description |
Check server connectivity by echoing the provided prompt message back; returns the prompt text as confirmation the server is reachable. |
help |
name_restates_behavior no_return_description |
Return usage instructions and descriptions of the available MCP tools on this server. |
version |
name_restates_behavior |
Return version and system details for the Codex CLI, Node.js runtime, platform, and MCP server components. |
brainstorm |
no_return_description |
Generate creative ideas for a challenge using structured frameworks (divergent, convergent, scamper, design-thinking, lateral, auto), optionally including feasibility and impact analysis. Returns the generated idea list plus the requested analysis. |
timeout-test |
name_restates_behavior no_return_description |
Verify timeout-prevention behavior by running for the specified duration in milliseconds; returns the outcome of the timed run. |
list-sessions |
name_restates_behavior |
List active Codex conversation sessions with metadata, or manage them via the action parameter (list, delete, clear). Returns the action performed along with session metadata and statistics. |
do-act |
params_unexplained |
Execute a task via Codex, validate the result with a shell command, and automatically retry with fixes when verification fails (Act-Check-Fix loop). localProvider selects the local inference backend to use (lmstudio or ollama). Returns status, attempt count, duration, and step details. |
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
ask-codex |
batch-codex |
medium | Both tools execute Codex on prompts, and a task phrased as 'use codex to handle these files' or 'run codex on a set of changes' could go either way since ask-codex also accepts multiple files via @syntax while batch-codex is only clearly indicated when the user explicitly asks to batch/delegate multiple atomic tasks; the shared 'codex' token and functional overlap create genuine selection ambiguity. |
list-sessions |
list-skills |
low | The definitions are semantically distinct (conversation sessions vs. available skills), so a natural-language task like 'show active sessions' or 'what skills are available' maps unambiguously; only the shared 'list' prefix produces lexical similarity, which is not enough for real confusion. |
Compare the field