01Safety
0.0 / 30
What changed in the harness
Selection accuracy 94→100, token cost up 18%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
20.4 / 30
03Economics
18.4 / 20
04Discoverability
12.5 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
24 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_pages |
no_return_description |
List the open tabs and return each one's index, title, and URL, with the currently selected tab marked so you can see which tab subsequent tab-level commands will act on. |
navigate_page |
no_return_description |
Navigate the currently selected tab to the given URL, loading the target page in the same tab and returning once the page has finished loading. |
select_page |
no_return_description |
Make the tab matching the given index, URL substring, or title substring the active (selected) tab that subsequent commands operate on; index takes precedence when more than one criterion matches. Returns once the selection has changed. |
close_page |
no_return_description |
Close the tab at the given page index, freeing that tab from the browser session, and return once the tab has been closed. |
take_snapshot |
no_return_description |
Capture a DOM snapshot of the current page, returning stable UIDs for elements so later commands can target them by UID, with optional text, attributes, and depth controls; retake the snapshot after any navigation because UIDs go stale. By default returns the snapshot inline up to maxLines, or saves the full snapshot to a file when saveTo is set. |
resolve_uid_to_selector |
name_restates_behavior no_return_description |
Turn a UID taken from a snapshot into a CSS selector that addresses the same element, which lets you reuse element targets in selector-based commands; the conversion fails if the UID is stale because the page changed after the snapshot. Returns the resolved CSS selector. |
clear_snapshot |
name_restates_behavior no_return_description |
Discard the cached DOM snapshot and its UIDs so stale elements can no longer be referenced; usually not needed, but do this before taking a fresh snapshot when the page has changed. Returns after the cache is cleared. |
click_by_uid |
no_return_description |
Click the element identified by the given UID, dispatching a normal click or a double-click when dblClick is set, to trigger the on-page interaction before reading the resulting state; returns once the click has been processed. |
hover_by_uid |
name_restates_behavior no_return_description |
Position the mouse over the element identified by the given UID to trigger its hover state, for example to reveal dropdowns or tooltips, before taking a snapshot or performing further interactions; returns once the hover is applied. |
fill_by_uid |
name_restates_behavior no_return_description |
Set the text of the input or textarea identified by the given UID to the supplied value, replacing whatever the field currently contains, so the form reflects the new value when read or submitted; returns after the value is entered. |
drag_by_uid_to_uid |
no_return_description |
Drag the element with fromUid onto the element with toUid, firing the HTML5 drag-and-drop events the page listens for so reordering and drop targets respond; returns when the drag completes. |
fill_form_by_uid |
no_return_description |
Fill several form fields in a single call by mapping each field's UID to the value to enter, which is more efficient than repeated fill_by_uid calls for multi-field forms; returns after all fields are filled. |
upload_file_by_uid |
name_restates_behavior no_return_description |
Attach the local file at filePath to the file input identified by the given UID so the chosen file is submitted with the form, for example after submitting the page; returns once the file is attached. |
screenshot_page |
name_restates_behavior no_return_description |
Take a screenshot of the whole visible page and return it as a base64-encoded PNG, or pass saveTo to write the image to a file and optionally preview it inline instead of receiving image data in the response. |
screenshot_by_uid |
name_restates_behavior no_return_description |
Take a screenshot of only the element identified by the given UID, cropped to that element's area, and return it as a base64-encoded PNG, or pass saveTo to write the image to a file instead of receiving inline image data. |
list_downloads |
name_restates_behavior no_return_description |
Return the downloads that Firefox has tracked since the server started, giving each download's status (in_progress, complete, or canceled) and the path where the file was saved; filter the listing by status, URL substring, or a maximum count, and choose text or JSON output. |
clear_downloads |
name_restates_behavior no_return_description |
Empty the buffer of tracked downloads so that subsequent list_downloads calls show only downloads that start after this point, which is useful between test runs to reset the view; returns after the buffer is cleared. |
set_download_behavior |
no_return_description |
Choose how downloads are handled: allowed saves files silently to the default download directory, denied cancels them, and default returns to the browser's normal handling, sidestepping the native save-file dialog during automation (requires a recent Firefox); returns once the behavior is applied. |
dismiss_dialog |
name_restates_behavior no_return_description |
Close the currently displayed browser dialog by answering it negatively (cancel), for example to get past an unexpected alert or confirmation so page scripts can continue; use accept_dialog instead when you need to confirm or supply prompt text. Returns once the dialog is dismissed. |
navigate_history |
name_restates_behavior no_return_description |
Move the selected tab one step backward or forward through its session history, like pressing the browser's back or forward button; note that UIDs from snapshots taken before this navigation become stale, so retake a snapshot afterwards. Returns once the page has loaded. |
set_viewport_size |
no_return_description |
Resize the browser viewport to the given width and height in pixels, which is useful for testing how the page's responsive layout behaves at different screen sizes or for controlling what is visible in screenshots; returns once the resize is applied. |
restart_firefox |
no_return_description |
Restart Firefox with a different configuration, closing all current tabs. You can change the binary path, profile path, environment variables in KEY=VALUE format, headless mode, start URL, and startup preferences (prefs values are auto-typed, and prefs requires MOZ_REMOTE_ALLOW_SYSTEM_ACCESS=1). Returns the result of the restart once the browser is available again. |
install_extension |
no_return_description |
Install a Firefox extension using the WebDriver BiDi webExtension.install command, either from an archive (.xpi/.zip), base64-encoded data, or an unpacked directory. Choose the source with the type parameter and provide the matching path or value; by default the extension is installed temporarily unless permanent is set (which requires a signed extension). Returns the extension ID, which can be passed to uninstall_extension or matched against list_extensions. |
evaluate_script |
no_return_description |
Execute a JS function in the page and return its result. Pass element UIDs from a snapshot as function arguments. Use saveTo to write the result to a file as JSON instead of returning it inline, and optionally preview characters of the saved result with the preview parameter. Prefer the UID tools for interactions. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
fill_by_uid |
fill_form_by_uid |
high | Both fill text inputs by UID and are semantically near-identical in descriptions ('Fill text input/textarea by UID' vs 'Fill multiple form fields at once'). A task like 'fill in the form' or 'fill the fields' could cause an agent to pick fill_by_uid when a single field is named or fill_form_by_uid when describing one input, selecting the wrong member. |
get_firefox_output |
get_firefox_info |
medium | Both return 'information about Firefox' in broad terms: one retrieves logs/output, the other retrieves instance configuration. A vague task like 'get info about what Firefox is doing' or 'check Firefox output' could plausibly make the agent pick the wrong one, since the output file location overlaps both descriptions. |
screenshot_page |
screenshot_by_uid |
medium | Both capture base64 PNG screenshots, differing only by scope (whole page vs element by UID). A task like 'take a screenshot of the search bar' or 'screenshot the form' requires resolving element vs page scope; without a UID context an agent could pick screenshot_page for an element-focused task or screenshot_by_uid for a full-page request. |
Compare the field