Public leaderboard

Public assessment

ymw0407/auth-fetch-mcp (auth-fetch-mcp)

auth-fetch-mcp · v3.0.3 · scanned

What changed in the harness

Selection accuracy 82→89, token cost up 11%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30
02Legibility

25.7 / 30

25.7 out of 30
03Economics

20.0 / 20

20.0 out of 20
04Discoverability

12.8 / 20

12.8 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool Defect types Suggested rewrite
download_media
no_return_description
Downloads one or more files (images, videos, or other media) from URLs — typically ones found in auth_fetch results — using the browser's saved session cookies for authentication, so no re-login is needed. Saves each file to output_dir (or ~/.auth-fetch-mcp/downloads/<timestamp>/ if omitted) and returns the local file paths and download status for each URL.
list_pages
no_return_description
Returns an array of all currently open browser tabs, each entry including its URL and page title, so callers can identify which tab to target for further actions.
close_browser
name_restates_behavior no_return_description
Shuts down the browser window used by auth_fetch and download_media, freeing resources when no more page captures or downloads are needed. Login sessions/cookies persist to disk and are automatically restored the next time a browser is opened. Returns a confirmation once the browser process has closed.

Selection evidence

Confusable tool pairs.

0 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A Tool B Confidence Why they collide
No confusable tool pairs were flagged in this assessment.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard