01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 7%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.2 / 30
03Economics
19.9 / 20
04Discoverability
12.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
7 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
create_diff |
no_return_description |
Shows a diff between an existing file's current content and the proposed newContent, and prompts the user to approve or reject it before applying. Only works on existing files (use a separate tool to create new ones). Returns whether the user approved the change and, if approved, applies it to the file on disk. |
open_file |
no_return_description |
Opens an existing file at the given path in the VS Code editor, optionally in a specific view column, in preview mode, or without stealing focus. Use this after create_diff or after creating a new file so the change is visible to the user. Returns confirmation that the file was opened, or an error if the path could not be found. |
open_project |
no_return_description |
Opens the given project folder in VS Code, optionally in a new window, and establishes it as the active working directory for the AI Agent session. Call this at the start of a new session. Returns confirmation of the opened project and its resolved path. |
check_extension_status |
no_return_description |
Checks whether the VS Code MCP Extension is installed and responding for the given project, useful for diagnosing connectivity issues before calling other tools. Returns a status object indicating whether the extension is reachable. |
get_extension_port |
name_restates_behavior no_return_description |
Looks up the local port the VS Code MCP Extension server is listening on for the given project, needed when a client must connect to the extension's HTTP API directly rather than through this MCP server. Returns the port number, or an error if the extension isn't running. |
list_available_projects |
no_return_description |
Reads the local port registry file to find all VS Code projects currently known to the MCP extension, so the user can pick which one to work with. Takes no parameters. Returns an array of available projects with their paths and assigned ports. |
get_context_tabs |
no_return_description |
Retrieves the tabs the user has explicitly marked for AI context via the context-toggle UI in VS Code, optionally narrowed to specific files and line ranges using the selections parameter. Returns the list of marked tabs, including file content by default (set includeContent to false to omit it). |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_active_tabs |
get_context_tabs |
high | Both retrieve info about open VS Code tabs for AI context; an agent asked to 'get the open tabs' or 'get context from the editor' could easily pick either, since the distinction (all open tabs vs. explicitly UI-marked tabs) is a subtle semantic detail not obvious from a casual task phrasing. |
open_file |
open_project |
medium | A vague instruction like 'open X in VS Code' is ambiguous between opening a single file (open_file) and opening a project/folder (open_project), especially if X could be interpreted as either a file or a working directory. |
check_extension_status |
get_extension_port |
low | Both check on the state of the VS Code extension, but their purposes (status check vs retrieving a port number) are distinct enough that only a very generic 'check the extension' request without specifics could cause confusion, and even then the outputs sought differ clearly. |
Compare the field