01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→95, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.2 / 30
03Economics
17.6 / 20
04Discoverability
12.1 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
wopee_send_chat_message |
no_return_description |
Send a message to the current project's chat room. Use this to post status updates (e.g., 'Test run started...', 'Analysis complete') or informational messages to the chat. The message will appear as a SYSTEM message in the chat room. Requires WOPEE_PROJECT_UUID to be configured. Returns a confirmation that the message was sent successfully, or an error if the message could not be delivered. |
wopee_create_github_issue |
no_return_description |
Create a new GitHub issue in the project's connected repository. Use this to report bugs found during testing, track test failures, or create action items from chat discussions. The issue will be created in the GitHub repository linked to the current project. Requires the project to have GitHub integration configured and WOPEE_PROJECT_UUID to be set. Returns the created issue's details (including its number and URL) on success, or an error message if creation fails. |
Selection evidence
Confusable tool pairs.
8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
wopee_dispatch_analysis |
wopee_dispatch_agent |
high | Both tools dispatch an autonomous browser-driving AI agent; a task like 'run the AI agent against my app' does not clearly distinguish crawling/mapping the application (dispatch_analysis) from executing specific test cases (dispatch_agent), since both take suite/app context and have side effects. |
wopee_create_blank_suite |
wopee_dispatch_analysis |
medium | Both create an analysis suite; a request like 'create an analysis suite for my web app' could pick the blank-suite tool (manual build) when dispatch_analysis creates AND populates the suite in one step, and neither name says 'crawl'. |
wopee_fetch_artifact |
wopee_generate_artifact |
medium | A task like 'I want the user stories / test artifacts for suite X' is ambiguous between retrieving existing artifacts (fetch) and having the AI author them (generate); generate is also what makes artifacts exist in the first place. |
wopee_update_artifact |
wopee_generate_artifact |
medium | Both write artifact content; 'create/update the user stories in my suite' could map to caller-supplied overwrite (update_artifact) or AI-authored generation (generate_artifact), and both work even on suites with no prior artifact. |
wopee_dispatch_analysis |
wopee_generate_artifact |
medium | A request like 'analyze my web app and produce the test artifacts' could either dispatch a crawl that creates and populates a suite (dispatch_analysis) or run artifact generation (generate_artifact), since generate_artifact's APP_CONTEXT/user-story types are also framed as analysis output. |
wopee_fetch_executed_test_cases |
wopee_fetch_test_inventory |
medium | Both return test cases with status; 'list the test cases and their status for A001' fits inventory's authoritative NOT_RUN view, while 'show me the executed test-case results for my suite' fits executed_test_cases — the surface area overlaps heavily despite one being per-suite. |
wopee_fetch_recent_executions |
wopee_fetch_test_inventory |
medium | A broad status request like 'what's the status of my tests / how did the tests go' is explicitly tuned for recent_executions, but intent like 'show me all my executed and not-run tests in one table' overlaps with inventory, so a vague status question can land on the wrong fetcher. |
wopee_fetch_executed_test_cases |
wopee_fetch_recent_executions |
medium | Both are read-only fetchers of execution results; 'get the results of the tests in suite X' needs executed_test_cases while 'what's the latest test status' needs recent_executions, and the per-suite-full vs project-wide-recent-20 distinction is easy to miss in a generic 'show me test results' request. |
Compare the field