Public assessment

auspy/supasidebar-mcp (supasidebar-mcp)

supasidebar-mcp · v0.2.0 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 96→98, token cost up 5%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

19.0 / 30

19.0 out of 30

03Economics

16.7 / 20

16.7 out of 20

04Discoverability

13.8 / 20

13.8 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

25 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`list_spaces`	name_restates_behavior no_return_description	List all spaces in SupaSidebar, returning each space's ID, name, color, and icon. Spaces are top-level collections that organize your links; use the returned IDs with list_links, list_folders, switch_space, and other space-scoped tools.
`search`	no_return_description	Search across all links in SupaSidebar by name, URL, notes, or tags using fuzzy matching. Returns matching links with their ID, name, URL, tags, and notes, up to `limit` results.
`list_folders`	name_restates_behavior no_return_description	List folders within a given space, returning each folder's ID, name, and parent folder ID for nested folders. Use the spaceId from list_spaces, and pass the returned folder IDs to list_links, create_folder, or move_link.
`list_tags`	no_return_description	List all tags used across your links, sorted by usage count. Returns each tag's name and the number of links using it.
`get_live_tabs`	no_return_description	Get currently open browser tabs, optionally filtered to a single browser (Arc, Chrome, Safari, Firefox, Edge, Brave, etc). Returns each tab's title, URL, and browser.
`list_recent`	no_return_description	List recently opened links, ordered by most recent first, returning each link's name, URL, and the time it was opened. If you don't see what the user wants in the first call, do NOT give up — increase `limit` (the default of 50 is small relative to a power user's history), page back with `offset`, or narrow with `day`/`since`/`until`. To find what someone opened on a specific day, prefer `day="YYYY-MM-DD"` (or `"today"`/`"yesterday"`) over scrolling. Combine `day` with a higher `limit` to see everything from that day.
`get_visibility`	no_return_description	Check whether the sidebar and command panel are currently visible, without changing their state. Returns booleans for sidebar visibility and command panel visibility. Use this instead of toggling just to check.
`launch_sidebar`	no_return_description	Launch the SupaSidebar app if it's not already running, using macOS's 'open -a' command. Returns whether the app was launched or was already running.
`toggle_command_panel`	no_return_description	Toggle the SupaSidebar command panel (search/command palette): opens it if closed, closes it if open. Returns the new open/closed state.
`switch_space`	no_return_description	Switch the active SupaSidebar space to the one with the given spaceId (get IDs from list_spaces). Returns the switched-to space's details.
`open_preferences`	no_return_description	Open the SupaSidebar Preferences window, optionally jumping to a specific tab (general, sidebar, mouseTriggers, shortcuts, spaces, liveTabs, voice, aiTags, importExport, commandPanel, airTrafficControl, refer, about). Returns confirmation that the window was opened.
`update_setting`	no_return_description	Change a SupaSidebar setting identified by key to the given value (use get_settings first to see available keys and current values). Returns the updated setting's new value.
`enable_feature`	no_return_description	Enable a named feature preset that configures multiple settings at once. Presets: 'Smart Attach' (auto-show with browser), 'Independent Mode' (always visible), 'Space Isolation' (separate tabs per space), 'Minimal Sidebar' (compact and clean), 'Full Featured' (everything on). Returns the list of settings that were changed.
`update_shortcut`	no_return_description	Change a keyboard shortcut binding to the given key and modifiers (at least one modifier key required). Use get_shortcuts to see available shortcut names. Returns the updated shortcut binding.
`clear_shortcut`	no_return_description	Remove a keyboard shortcut binding so it no longer has a key combination assigned. Returns confirmation that the binding was cleared.
`web_search`	no_return_description	Search the web using a search engine or custom website shortcut, opening the results in a browser. Built-in engines: Google, Bing, DuckDuckGo, Yahoo, Perplexity, Brave, Kagi. Custom shortcuts let you search specific sites (e.g. YouTube, Reddit, GitHub) — use list_search_shortcuts to see all available.
`remove_search_shortcut`	name_restates_behavior no_return_description	Delete a custom website search shortcut identified by its keyword or ID, so it can no longer be used with web_search. Returns confirmation that the shortcut was removed.
`create_space`	name_restates_behavior no_return_description	Create a new space in SupaSidebar with the given name, color, and icon. Returns the newly created space's ID and details, which can be used with list_links, create_folder, add_link, and switch_space.
`create_folder`	no_return_description	Create a new folder inside a space (use list_spaces for space IDs, list_folders for parent folder IDs). Returns the newly created folder's ID and details.
`add_link`	no_return_description	Save a new link to SupaSidebar. If name is omitted, the title is fetched from the page automatically. Returns the newly saved link's ID and details.
`move_link`	no_return_description	Move an existing link (get its ID from list_links or search) to a different space or folder. Pass targetFolderId as null to move to root (unfiled). Returns the link's updated location.
`list_atc_rules`	no_return_description	List all Air Traffic Control (ATC) rules, which automatically route URLs to specific spaces (save rules) or browsers/profiles (open rules), evaluated top-to-bottom with first match winning. Returns each rule's ID, name, type, match pattern, and target.
`add_atc_rule`	no_return_description	Create a new Air Traffic Control rule: save rules route saved links to a specific space, open rules open matching URLs in a specific browser/profile. Use list_browser_profiles for profile IDs and list_spaces for space IDs. Returns the newly created rule's ID and details.
`delete_atc_rule`	no_return_description	Delete an ATC rule by ID (get IDs from list_atc_rules). Returns confirmation that the rule was deleted.
`list_browser_profiles`	no_return_description	List browser profiles discovered by SupaSidebar, returning each profile's ID, browser name, and profile name. Use this to get profile IDs for ATC rules that route URLs to a specific browser profile (e.g. 'Safari Work profile').

Selection evidence

Confusable tool pairs.

11 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`search`	`web_search`	high	A bare request like 'search for X' is ambiguous between searching saved links (search) and searching the web (web_search); the tool names alone don't disambiguate intent.
`toggle_sidebar`	`launch_sidebar`	high	'Open the sidebar' is ambiguous when the app state is unknown — it could mean start the app (launch_sidebar) or show it if already running (toggle_sidebar), and an agent without state info may pick the wrong one.
`list_browser_profiles`	`list_installed_browsers`	medium	'What browsers/profiles do I have' phrasing can blur the distinction between installed browser apps and configured profiles within them, especially when the user just says 'show my browsers'.
`add_atc_rule`	`update_atc_rule`	medium	A request like 'route x.com to my Work space' could be creating a new rule or modifying an existing unnamed one; without checking list_atc_rules first an agent might default to add instead of update or vice versa.
`get_settings`	`get_shortcuts`	medium	Keyboard shortcuts are technically settings, so 'show me the shortcut settings' or 'what's my toggle sidebar setting' could plausibly route to either get_settings(key='shortcut...') or get_shortcuts.
`update_setting`	`update_shortcut`	medium	Both tools 'update' configuration; a request like 'change my toggle sidebar setting to cmd+k' blurs whether it's a generic setting (update_setting) or a keybinding (update_shortcut).
`update_shortcut`	`clear_shortcut`	medium	'Disable/remove the shortcut for X' could be interpreted as unbinding it entirely (clear_shortcut) or reassigning it to no meaningful key via update_shortcut, which actually requires a key+modifier so clear is likely intended but confusable.
`toggle_sidebar`	`toggle_command_panel`	medium	'Open the search panel' or generic 'toggle it' phrasing could be misapplied to the sidebar instead of the command panel since both are UI toggle actions sharing 'sidebar'/'supa' framing in the app.
`list_links`	`list_folders`	medium	'Show me what's in my Work space' is ambiguous between listing links and listing folders, since both are valid interpretations of space contents.
`add_link`	`move_link`	low	'Put this link in my Work space' could mean saving a new link there (add_link) or relocating an existing saved link (move_link) if the agent isn't sure the link is already saved.
`create_space`	`create_folder`	low	A vague request like 'create a new collection for X' could map to either a top-level space or a folder within one, depending on unclear hierarchy intent.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard