01Safety
0.0 / 30
What changed in the harness
Selection accuracy 91→96, token cost up 4%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.6 / 30
03Economics
17.1 / 20
04Discoverability
12.7 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
ask-codex |
no_return_description |
Execute Codex CLI with file analysis (@syntax), skills ($syntax), model selection, and safety controls; supports changeMode. Returns the CLI's response text, or, when changeMode is enabled, structured OLD/NEW edit blocks (optionally chunked for large outputs, retrievable via fetch-chunk). |
review-changes |
no_return_description |
Run a code review against the current repository using Codex CLI's native review subcommand, targeting uncommitted changes, a base-branch diff, or a specific commit. Returns the review findings and summary as text. |
ping |
no_return_description |
Echo the given prompt back verbatim, used to verify the MCP server is reachable and responding. Returns the same text that was sent in the prompt parameter. |
help |
name_restates_behavior no_return_description |
List the available Codex MCP tools, their usage patterns, and configuration tips for this server. Returns formatted help text as a string. |
version |
name_restates_behavior |
Report the installed Codex CLI version alongside the Node.js runtime version, host platform, and MCP server version, useful for diagnosing compatibility issues. |
brainstorm |
no_return_description |
Generate creative ideas using structured frameworks (divergent, convergent, SCAMPER, design-thinking, lateral) with domain context and constraints. Returns a formatted list of generated ideas, optionally including feasibility/impact analysis for each. |
timeout-test |
name_restates_behavior no_return_description |
Simulate a long-running Codex operation for a configurable duration to verify the MCP server's timeout-prevention mechanism keeps the connection alive. Returns a completion status once the specified duration has elapsed. |
do-act |
params_unexplained |
Execute a task via Codex, verify the result by running a shell command, and automatically retry with fixes on failure (act-check-fix loop). Set localProvider to 'lmstudio' or 'ollama' to choose which local model backend to use when oss is enabled. Returns the final status, number of attempts, total duration, and a step-by-step execution log. |
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
ask-codex |
batch-codex |
medium | A task like 'apply this fix across multiple files' could be phrased as a single ask-codex prompt referencing multiple @files, or as several atomic batch-codex tasks — the agent may not clearly decide whether to decompose into a task array or issue one combined prompt, especially since both support @-file targeting and model overrides. |
list-sessions |
list-skills |
low | Both are simple listing tools but target unrelated resources (conversation sessions vs. discoverable skills in .agents/skills/); a vague request like 'what's available' could momentarily cause hesitation, but the distinct terms 'session' and 'skill' make genuine misselection unlikely. |
Compare the field