Public leaderboard

Public assessment

doctorm333/promptpilot-mcp-server (promptpilot-mcp)

promptpilot-mcp · v1.0.0 · scanned

What changed in the harness

Selection accuracy 100→100, token cost up 10%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30
02Legibility

26.3 / 30

26.3 out of 30
03Economics

19.8 / 20

19.8 out of 20
04Discoverability

9.9 / 20

9.9 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

6 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool Defect types Suggested rewrite
list_models
name_restates_behavior no_return_description
Query the Pollinations catalog of supported generation models across image, video, and audio, including each model's type and pricing. Use it to select a valid model ID and understand its cost before generating; returns the full list of available models with their type and pricing information.
list_styles
name_restates_behavior no_return_description
Retrieve the complete set of prompt-building vocabulary grouped by category, covering styles, lighting, camera angles, moods, colors, and quality tags. Use it to discover valid tag options for build_prompt; returns the categorized list of all available tags.
build_prompt
name_restates_behavior no_return_description
Combine a required subject with optional style, lighting, camera, mood, color, and quality tag selections into a single optimized prompt string. Call this before a generation tool to ensure a well-formed prompt; returns the assembled prompt text ready for generate_image or generate_video.
generate_video
params_unexplained
Generate a video clip via the Pollinations API and return a URL to the result. aspect_ratio sets the video frame ratio as width:height, choosing one of 1:1, 16:9, or 9:16; duration specifies length in seconds, which may be capped depending on the chosen model; seed makes output reproducible.
check_balance
name_restates_behavior no_return_description
Inspect the Pollinations API credit balance (pollen) to see remaining credits for paid models. Query this before running paid generation workloads to avoid failures when credits run out; returns the current balance with remaining credit details.
generate_batch
no_return_description
Generate up to 10 images in a single request, which is more efficient than calling generate_image repeatedly. Takes an array of up to 10 prompt objects, each with a required prompt and optional model, width, height, and seed; returns the generated image URLs for the batch.

Selection evidence

Confusable tool pairs.

5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A Tool B Confidence Why they collide
generate_image generate_batch medium A task like 'generate 5 images of cats' fits both tools: generate_batch exists precisely to avoid calling generate_image repeatedly, but an agent may pick generate_image and loop, or pick generate_batch for a single-image intent. The neighbor-the-boundary use case makes the choice genuinely ambiguous.
generate_image generate_video medium Tasks that omit the medium ('make a clip of a whale', 'create an animation of a dancing robot', 'generate a scene of a sunset') don't clearly say image vs video, so an agent could pick either generate tool despite the descriptions distinguishing them.
list_models list_styles medium Both are zero-arg listing tools, so vague discovery tasks like 'what can you generate?', 'show me all available options', or 'what are my choices?' could plausibly resolve to either tool depending on whether the agent interprets 'options' as models or styles.
list_styles build_prompt medium Tasks about prompt construction with style options ('I need a prompt with cinematic lighting and mood', 'help me pick options for a prompt', 'give me a dramatic, colorful prompt') overlap heavily in vocabulary and could be answered by listing style options or by building the prompt itself.
generate_image generate_audio low Tasks like 'generate a tone' or 'generate a voice clip' are usually medium-specific, but words like 'clip', 'sound' misused, or a task asking to 'generate a bird' for a song vs a picture creates a small but real chance of confusion.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard