Public assessment

shipstatic/mcp (@shipstatic/mcp)

shipstatic-mcp · v0.6.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 4%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

25.3 / 30

25.3 out of 30

03Economics

19.7 / 20

19.7 out of 20

04Discoverability

15.2 / 20

15.2 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

10 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`deployments_list`	name_restates_behavior	List all deployments you have created, including each deployment's live URL, status, labels, and password protection state, so you can find a deployment's hostname or check its current state.
`deployments_get`	name_restates_behavior	Fetch full details for a single deployment by hostname, including its live URL, status, file count, size, labels, and password protection state, so you can inspect a specific deployment after listing your deployments.
`deployments_set`	no_return_description	Update the labels of a deployment by hostname, replacing all existing labels with the labels you provide (pass an empty array to clear all labels). Returns confirmation that the labels were updated.
`deployments_remove`	no_return_description	Permanently delete a deployment and its files by hostname. You MUST confirm with the user before calling this tool, referencing the deployment. Returns confirmation that the deployment was deleted.
`domains_set`	no_return_description	Create or update a custom domain. Can reserve a name (omit deployment), link it to a deployment, switch deployments, or update labels. After creating, call domains_records and show the DNS records to the user. Returns the created or updated domain details.
`domains_list`	name_restates_behavior	List all custom domains you have created, including each domain's URL, linked deployment, and verification status, so you can find a domain's name or see how it is currently set up.
`domains_get`	name_restates_behavior	Fetch full details for a single custom domain by name, including its URL, linked deployment, verification status, and labels, so you can inspect a specific domain after listing your domains.
`domains_dns`	no_return_description	Look up the DNS provider for a domain (e.g. Cloudflare, Namecheap) so the user knows where to configure their DNS records. Returns the name of the DNS provider for the given domain.
`domains_verify`	no_return_description	Trigger DNS verification for a custom domain. Call after the user has configured the DNS records from domains_records. Returns the current verification status; verification is asynchronous, so the domain status updates once DNS propagates.
`domains_remove`	no_return_description	Permanently delete a custom domain by name. You MUST confirm with the user before calling this tool, referencing the domain name. Returns confirmation that the domain was deleted.

Selection evidence

Confusable tool pairs.

10 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`deployments_list`	`domains_list`	medium	Both return a list of URLs with status for resources users often call 'sites'; a task like 'show me all my URLs and their status' doesn't reveal whether deployments or domains is meant.
`deployments_get`	`domains_get`	medium	Both accept a hostname-like identifier and return URL, status, and labels. A task such as 'get the status and labels for my site' could lead the agent to the wrong resource, since users rarely say 'deployment' vs 'domain'.
`domains_list`	`domains_get`	medium	Both report a domain's linked deployment and verification status; a task like 'what's the verification state of my domains' is ambiguous between scanning all (list) and inspecting one (get).
`deployments_list`	`deployments_get`	medium	Both descriptions share URL, status, labels, and password-protection fields; a task like 'check the password protection or labels on my deployment' doesn't specify whether all deployments or a specific one is wanted.
`deployments_remove`	`domains_remove`	medium	Both permanently delete with confirmation, and a user saying 'take down/delete my site' could mean the deployment or the custom domain pointing at it; both match the delete wording.
`deployments_set`	`domains_set`	medium	Both accept labels and describe themselves as updates; a task like 'update the labels on my production resource' gives no clue whether the labels belong to a deployment or a domain.
`domains_records`	`domains_share`	medium	domains_share is described as returning the required DNS records (as a shareable hash), so a task like 'get the DNS records I need to configure' could plausibly select either tool.
`domains_dns`	`domains_verify`	low	A task like 'check the DNS for my domain' is ambiguous between looking up the DNS provider (dns) and triggering DNS verification (verify).
`domains_dns`	`domains_share`	low	A vague request like 'help me with DNS setup for my domain' could match either the provider lookup or the shareable DNS hash tool.
`domains_get`	`domains_records`	low	A task like 'show me domain info for example.com' could select get (details) when the user actually wants the DNS records to configure; both mention the domain and 'get'.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard