01Safety
0.0 / 30
What changed in the harness
Selection accuracy 94→94, token cost up 8%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.8 / 30
03Economics
19.5 / 20
04Discoverability
15.5 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
4 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
brainstorm |
no_return_description |
Run a multi-round brainstorming debate between multiple AI models. IMPORTANT: Before calling, you MUST present the user with the two mode options and ask them to choose: 1. API mode - uses external API keys to call models (OpenAI, Gemini, DeepSeek, etc.), best when API keys are configured. 2. Hosted mode - no API keys needed; you execute prompts via sub-agents with models available in your environment, and the same model can be used multiple times for different perspectives. Proceed based on the choice: for API mode set mode='api' (with participate=true, the default, you also participate as a debater via brainstorm_respond); for hosted mode set mode='hosted', ask which models to use, spawn a sub-agent for each to run the prompts, collect the responses, and pass them to brainstorm_collect. Returns the session_id for the debate and the external models' responses for the current round, which you should engage with using brainstorm_respond; rounds continue automatically until a synthesis or final formatted result is produced. |
brainstorm_respond |
no_return_description |
Submit your (Claude's) substantive contribution for the current round of an interactive brainstorm session. Call this after the brainstorm tool returns the external models' responses; read them carefully and engage with specific points - agree, disagree, build on, or challenge ideas rather than just summarizing. session_id identifies the session returned by the brainstorm tool; response is your contribution of at least 50 characters. Returns the next round of external-model responses to continue the debate, or the final synthesis/result after the last round, which runs automatically once your response is submitted. |
list_providers |
name_restates_behavior no_return_description |
Returns the AI providers currently configured for brainstorming so you can confirm which models are available and which providers need setup before running a brainstorm. The returned list covers each provider's name, API base URL, environment-variable name holding its API key (or NONE), and default model, matching how providers are registered via add_provider. Use this to check configuration when switching between API and hosted mode. |
add_provider |
name_restates_behavior no_return_description |
Register a new OpenAI-compatible AI provider so it can be used in brainstorming. Supply the provider's name, its API base URL, the environment-variable name that holds its API key (use 'NONE' if no key is required), and the default model to call. After registration the provider becomes available as a model source (e.g. 'provider:model') for the models and synthesizer parameters of the brainstorm tools. Returns a confirmation that the provider was added, or an error if the configuration is invalid. |
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
brainstorm |
brainstorm_quick |
medium | Both are entry-point brainstorming tools that accept a topic and return multi-model perspectives; the only differentiator in descriptions is round count/speed. A task like 'get diverse perspectives on this idea' or 'brainstorm this topic' without a time/rounds cue is ambiguous, especially since brainstorm_quick markets itself as 'diverse perspectives fast' while brainstorm pushes a heavier mode-asking flow. |
brainstorm_respond |
brainstorm_collect |
medium | Both 'submit responses for a brainstorm session' and share session/submit/response tokens. A task like 'submit the model responses for the ongoing brainstorm' is ambiguous about role: respond is Claude's own 50+ char contribution while collect takes an array of sub-agent responses; the agent may pick the wrong submission path when the user doesn't clarify whether Claude is a participant or host. |
Compare the field