01Safety
0.0 / 30
What changed in the harness
Selection accuracy 95→95, token cost up 16%, unconfirmed writes 0%→0%.
Category breakdown
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
0.0 / 30
24.6 / 30
20.0 / 20
11.5 / 20
Highest-impact fix
Estimated gain +30 pointsExpose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
12 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
browser_navigate |
no_return_description |
Navigate the current browser tab to the given URL and wait for the page to load. Returns the resulting page state (e.g. URL/title or an accessibility snapshot) after navigation completes. |
browser_go_back |
no_return_description |
Navigate the browser back to the previous page in history. Returns the resulting page state (e.g. URL/title) after the navigation completes. |
browser_go_forward |
no_return_description |
Navigate the browser forward to the next page in history. Returns the resulting page state (e.g. URL/title) after the navigation completes. |
browser_snapshot |
no_return_description |
Capture an accessibility snapshot of the current page, returning a structured tree of elements with reference IDs that can be used to target elements for interaction (click, type, hover, etc.). |
browser_click |
no_return_description |
Perform a mouse click on the specified element identified via a page snapshot reference. Returns confirmation of the click along with the resulting page state or updated snapshot. |
browser_hover |
no_return_description |
Move the mouse to hover over the specified element identified via a page snapshot reference, triggering any hover-based UI (tooltips, menus). Returns confirmation of the action along with the resulting page state. |
browser_type |
no_return_description |
Type the given text into the specified editable element identified via a page snapshot reference, optionally pressing Enter afterward if submit is true. Returns confirmation of the input along with the resulting page state. |
browser_select_option |
no_return_description |
Select one or more option values in the specified dropdown element identified via a page snapshot reference. Returns confirmation of the selection along with the resulting page state. |
browser_press_key |
no_return_description |
Simulate pressing a single keyboard key or character (e.g. 'ArrowLeft', 'a') on the currently focused element. Returns confirmation that the key press was dispatched along with the resulting page state. |
browser_wait |
no_return_description |
Pause execution for the specified number of seconds before continuing, useful for waiting on page transitions or animations. Returns confirmation once the wait period has elapsed. |
browser_get_console_logs |
name_restates_behavior no_return_description |
Retrieve the browser's console output (logs, warnings, and errors) accumulated since the page loaded or since logs were last cleared. Returns an array of console log entries with their severity level and message text. |
browser_screenshot |
no_return_description |
Capture a screenshot image of the current state of the browser page. Returns the screenshot as image data (e.g. base64-encoded PNG). |
Selection evidence
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
browser_go_back |
browser_go_forward |
high | Both take no parameters and mean directional page navigation; a vague task like 'go to the previous page' vs 'return to where I was' can be misread, and 'back'/'forward' semantics are easy to invert when inferred from ambiguous phrasing (e.g. 'undo that navigation'). |
browser_snapshot |
browser_screenshot |
high | Both capture the current page state; a task like 'show me the page' or 'capture what's on screen' could be satisfied by either, but snapshot returns accessibility refs for interaction while screenshot returns a visual image—agent may pick the wrong one depending on whether it conflates 'look at the page' with 'get element references'. |
browser_click |
browser_hover |
medium | Both act on an element via element+ref and are used to 'interact with' something; a task like 'trigger the dropdown menu on the nav item' could be satisfied by hover (to reveal) or click depending on site behavior, and phrasing like 'select that button' is ambiguous between hovering to inspect and clicking to activate. |
browser_click |
browser_type |
low | Both target an element via ref for interaction, but click has no text parameter while type requires text and submit; a task like 'fill in and submit the search box' is unambiguous enough that confusion is unlikely, though 'interact with the field' phrasing could momentarily blur the choice. |
Compare the field