Public leaderboard

Public assessment

123Ergo/unphurl-mcp (@unphurl/mcp-server)

unphurl-mcp-server · v0.2.0 · scanned

What changed in the harness

Selection accuracy 100→98, token cost down 1%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30
02Legibility

28.1 / 30

28.1 out of 30
03Economics

18.1 / 20

18.1 out of 20
04Discoverability

14.6 / 20

14.6 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

5 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool Defect types Suggested rewrite
signup
params_unexplained
Create a new Unphurl account. Returns an API key that is shown only once, so store it securely; the key will not work for URL checks until the user verifies their email by clicking the verification link sent to that email address (the link expires after 24 hours; use the "resend_verification" tool to request a new one). A new account starts with 20 free pipeline check credits for testing with real URLs; lookups of known domains (e.g. google.com, github.com) and cached domains are always free, and more credits can be purchased via the "purchase" tool. Once you have the key, set it as UNPHURL_API_KEY in the MCP server configuration. This tool does not require an API key. Parameters: email is the address where the verification link is sent; first_name is used for personalized emails; company is the organization the account belongs to (optional, used to label the account).
create_profile
no_return_description
Create a new custom scoring profile, or fully replace an existing profile with the same name (full replacement, not a merge). Profiles are sparse overrides: only specify weights for the signals you want to change; every other signal keeps its default weight. Weights are points added to the score when that signal fires; they do not need to total 100 (a total below 100 caps the maximum possible score, above 100 pushes signals toward the 100 cap quickly). See show_defaults for all 25 signals with their default weights, and use check_url or check_urls with the "profile" parameter to apply this profile. Returns the created or updated profile. Limit: 20 profiles per account; the name "default" is reserved. Example use-case profiles: cold-email (parked 30, chain_incomplete 25, ssl_invalid 15, brand_impersonation 10), security-bot (brand_impersonation 40, domain_age_7 30, redirects_5 25), lead gen (parked 35, http_only 20, chain_incomplete 20), SEO audit (redirects_5 30, chain_incomplete 30, parked 25). See the Unphurl API documentation for all 19 use-case weight examples.
delete_profile
no_return_description
Permanently delete a custom scoring profile. Any future check requests that reference this profile name will fall back to default weights. Run list_profiles first to see which profiles exist before deleting. Returns a confirmation once the profile has been deleted.
add_to_allowlist
no_return_description
Add one or more registrable domains to this account's trusted allowlist. Allowlisted domains suppress the compound signal and the brand impersonation floor in scoring, but the full pipeline still runs so all signals remain visible for monitoring (e.g. SSL expiry, parking). Submit only the registrable domain (e.g. partnerco.com); subdomains and full URLs are rejected, and adding partnerco.com automatically covers sub.partnerco.com and all other subdomains. Limits: 1,000 domains per account and 100 per request; duplicates are silently skipped. Returns a confirmation of which domains were added to the allowlist.
remove_from_allowlist
no_return_description
Remove one or more registrable domains from this account's trusted allowlist. Once removed, those domains resume normal scoring on the next check. Use list_allowlist to see what is currently on the list before removing. Returns a confirmation that the domains were removed from the allowlist.

Selection evidence

Confusable tool pairs.

4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A Tool B Confidence Why they collide
check_url check_urls medium Both tools perform URL safety checks and share nearly identical billing/scoring descriptions; a task phrased as 'check these URLs' with an unspecified count could lead an agent to pick the single-URL tool instead of the batch tool, and the singular/plural distinction is easy to miss when a user names multiple domains
get_balance get_stats low get_stats explicitly returns 'credit balance' among its fields, so a plain query like 'how many credits do I have?' could map to either tool; only tool descriptions hint that get_balance is the narrower, dedicated balance lookup
get_balance get_pricing low Both are credit-related queries sharing 'check/credit/pipeline' tokens; ambiguous phrasing like 'how much credit do I have/can I get' pits remaining-credits against package-pricing, though descriptions clearly separate 'remaining' from 'packages and prices'
get_pricing purchase low A task like 'what are my options for getting more credits' or 'set me up with credits' could route to either the pricing list or the purchase tool, since both describe credit packages; the descriptions do clarify price display vs Stripe checkout, so confusion is limited

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard