Public assessment

sh6drack/zen-mcp (zen-mcp)

zen-mcp · v1.2.2 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 87→85, token cost up 14%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

20.3 / 30

20.3 out of 30

03Economics

19.7 / 20

19.7 out of 20

04Discoverability

9.8 / 20

9.8 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

18 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`zen_list_pages`	name_restates_behavior no_return_description	Returns the list of currently open tabs in Zen Browser, each with its index, URL, and title, for use with zen_select_page or zen_close_tab.
`zen_select_page`	no_return_description	Sets the page/tab at the given index (as returned by zen_list_pages) as the active target for subsequent commands. Returns the newly active page's URL and title.
`zen_new_tab`	no_return_description	Opens a new browser tab, optionally navigating it to the given url (defaults to about:blank), and returns the new tab's index, URL, and title.
`zen_navigate`	no_return_description	Navigates the active page to the given url and returns the resulting page title and URL once loading completes.
`zen_snapshot`	no_return_description	Captures a structured snapshot of the active page's DOM, returning each visible element's role, text, and CSS selector. Use filter to scope results to "all", "interactive", or "form" elements (default: all), and selector to limit the snapshot to a subtree of the page.
`zen_screenshot`	no_return_description	Captures a screenshot of the current page and returns the image data.
`zen_click`	params_unexplained no_return_description	Clicks the element matching the given CSS selector (e.g. "#submit" or ".btn-primary") and returns whether the click succeeded.
`zen_fill`	no_return_description	Clears any existing content and fills a text input or textarea matched by the CSS selector with the given value, dispatching input/change events for framework compatibility, and returns confirmation the fill succeeded.
`zen_select_option`	no_return_description	Selects an option in the <select> element matched by the CSS selector, matching by option value or visible text, and returns the option that was selected.
`zen_check`	params_unexplained no_return_description	Checks or unchecks the checkbox, or selects the radio button, matched by the given CSS selector (checked defaults to true), and returns the resulting checked state.
`zen_get_form_fields`	no_return_description	Scans the active page and returns every form field's name, type, label, current value, and CSS selector, for use with zen_fill or zen_fill_form.
`zen_fill_form`	params_unexplained no_return_description	Fills multiple form fields in one call. Each entry in fields specifies a CSS selector, the value to apply, and an action (fill/select/check/uncheck/click; defaults to fill) to perform on that field. Returns the result of each field operation.
`zen_scroll`	params_unexplained no_return_description	Scrolls the page in the given direction ("up", "down", "top", or "bottom") by amount pixels (default: 500), or scrolls a specific element into view via selector, and returns the resulting scroll position.
`zen_wait`	no_return_description	Pauses execution for the given number of milliseconds (default: 1000) and returns once the wait completes.
`zen_get_page_text`	name_restates_behavior no_return_description	Returns the active page's title, URL, and visible text content (truncated to maxLength characters, default 8000), for quickly understanding what is on the page without a full snapshot.
`zen_close_tab`	no_return_description	Closes the tab at the given index (from zen_list_pages); fails if it is the last remaining tab. Returns confirmation that the tab was closed.
`zen_press_key`	no_return_description	Sends a keyboard event to the active page via BiDi input.performActions, optionally held with modifier keys (ctrl/shift/alt/meta). Supports special keys like Enter, Tab, Escape, Backspace, and arrow keys; for typing text use zen_fill instead. Returns confirmation that the key was dispatched.
`zen_reconnect`	no_return_description	Forces a disconnect and reconnect to Zen Browser; use when the connection is in a bad state or after restarting Zen. Returns the reconnected session status.

Selection evidence

Confusable tool pairs.

6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`zen_wait`	`zen_wait_for`	high	Both wait for something to happen; an agent told to 'wait for the page to load' or 'wait until the button appears' could pick zen_wait (fixed delay) instead of zen_wait_for (condition-based), especially since names differ only by a suffix.
`zen_fill`	`zen_fill_form`	medium	A task like 'fill in the login fields' could be done via repeated zen_fill calls or a single zen_fill_form call; the agent must decide between single-field and multi-field tools, and naming overlap makes the choice non-obvious.
`zen_select_page`	`zen_select_option`	medium	Both use 'select' plus an index/value parameter; a vague instruction like 'select the second option' could be misapplied to zen_select_page (tab selection) versus zen_select_option (dropdown selection) if context about tabs vs dropdowns is thin.
`zen_get_form_fields`	`zen_fill_form`	low	Both relate to forms; a task like 'fill out the form' might cause an agent to call zen_get_form_fields (inspection) when it actually needs zen_fill_form (action), though descriptions are distinct enough to usually disambiguate.
`zen_list_pages`	`zen_get_page_text`	low	Both return page/title info; 'what page am I on' style queries could plausibly trigger either zen_list_pages (all tabs) or zen_get_page_text (current page content), though scopes differ (all tabs vs current page text).
`zen_select_page`	`zen_close_tab`	low	Both take a tab index from zen_list_pages; a loosely worded 'switch away from that tab' instruction could be confused between selecting vs closing, though the actions are semantically distinct enough to rarely cause real confusion.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard