Public assessment

shipstatic/mcp (@shipstatic/mcp)

shipstatic-mcp · v0.6.0 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 6%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

23.7 / 30

23.7 out of 30

03Economics

19.7 / 20

19.7 out of 20

04Discoverability

15.8 / 20

15.8 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

12 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`deployments_list`	name_restates_behavior no_return_description	Fetches every deployment on the account, useful for finding a deployment hostname to pass to other tools. Returns each deployment's hostname/URL, status, labels, and password protection state.
`deployments_get`	name_restates_behavior no_return_description	Fetches the current state of a single deployment identified by hostname. Returns its URL, status, file count, size, labels, and password protection state.
`deployments_set`	no_return_description	Updates a deployment's labels, replacing the entire existing label set with the ones provided (pass an empty array to clear all labels). Returns the updated deployment record.
`deployments_remove`	no_return_description	Permanently deletes a deployment and all of its files; this cannot be undone. You MUST confirm with the user before calling this tool, referencing the deployment. Returns confirmation that the deployment was removed.
`domains_set`	no_return_description	Creates or updates a custom domain: reserve a name by omitting deployment, link or switch it to a deployment, or update its labels. After creating, call domains_records and show the DNS records to the user. Returns the created or updated domain record, including its verification status.
`domains_list`	name_restates_behavior no_return_description	Fetches every custom domain on the account, useful for finding a domain name to pass to other tools. Returns each domain's URL, linked deployment, and verification status.
`domains_get`	name_restates_behavior no_return_description	Fetches the current state of a single custom domain by name. Returns its URL, linked deployment, verification status, and labels.
`domains_dns`	no_return_description	Looks up which DNS provider (e.g. Cloudflare, Namecheap) currently hosts a domain's DNS, so the user knows where to configure records. Returns the detected provider name.
`domains_share`	no_return_description	Generates a shareable DNS setup hash for a domain so the user can view its required DNS records without needing an API key. Returns the shareable hash/link.
`domains_verify`	no_return_description	Triggers asynchronous DNS verification for a custom domain after the user has configured the records from domains_records; the domain's status updates once DNS propagates. Returns the verification status as of the call.
`domains_remove`	no_return_description	Permanently deletes a custom domain; this cannot be undone. You MUST confirm with the user before calling this tool, referencing the domain name. Returns confirmation that the domain was removed.
`whoami`	no_return_description	Fetches identity and billing info for the currently authenticated account. Returns the account email, plan tier, and current usage.

Selection evidence

Confusable tool pairs.

8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`deployments_list`	`domains_list`	medium	A vague request like 'show me all my sites' or 'list everything I've deployed' doesn't specify deployment vs custom domain, and both tools list all items with URL/status with no parameters, making the choice depend on terminology the user may not use precisely.
`deployments_get`	`domains_get`	medium	Both fetch details (URL, status, labels) for a single named entity via a string identifier; a request like 'get details for example.com' could be interpreted as a domain, but if the user is loosely referring to a site by its custom URL, the agent may not immediately know whether to call domains_get or resolve it as a deployment.
`deployments_remove`	`domains_remove`	medium	Both are near-identical 'permanently delete X, confirm first' tools differing only in entity type; a task like 'delete example.com' is ambiguous between deleting a custom domain record and deleting a deployment that happens to be reachable at that URL.
`deployments_set`	`domains_set`	medium	Both tools 'set'/update labels on an entity given a name; a request like 'label example.com as production' could plausibly target domains_set (labels field) or be mistaken for a deployment labeling task if the agent isn't sure which entity type the name refers to.
`domains_dns`	`domains_share`	medium	Both return DNS-related information a user can pass along to someone else (provider name vs. a shareable hash of records); a task like 'send me something I can give my DNS admin' is ambiguous between looking up the provider and generating a share link.
`domains_dns`	`domains_verify`	medium	Both are 'check the DNS' style actions; a task like 'check the DNS for my domain' could be read as looking up the DNS provider (domains_dns) or triggering verification of DNS propagation (domains_verify), since neither name makes the distinct action obvious.
`domains_get`	`domains_records`	medium	Both take a single domain and return domain-related info; a request like 'show me the DNS info for example.com' could trigger domains_get (verification status/labels) or domains_records (the actual DNS records to configure), since 'DNS info' maps loosely to both.
`domains_records`	`domains_share`	high	Both surface the same underlying DNS records for a domain — one directly, one via a shareable hash link — so a task like 'give me the DNS records to share with my team' is genuinely ambiguous between returning domains_records directly or generating a domains_share link.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard