Public leaderboard

Public assessment

microservices-sh/mcp (@microservices-sh/mcp)

microservices-sh-mcp · v0.1.3 · scanned

What changed in the harness

Selection accuracy 100→100, token cost up 2%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30
02Legibility

24.3 / 30

24.3 out of 30
03Economics

19.0 / 20

19.0 out of 20
04Discoverability

10.7 / 20

10.7 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

16 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool Defect types Suggested rewrite
list_templates
no_return_description
List available microservices.sh app templates and return a list of templates with their ids and summaries, so you can choose a foundation for a new Cloudflare app. Use this before inspect_template, compose_app, generate_project, or run_checks.
inspect_template
no_return_description
Inspect one app template contract and return its details, including supported modules, default configuration, runtime metadata, and generation behavior. Use this before composing or generating a project from a specific template id.
list_modules
no_return_description
List available verified microservices.sh modules and return a list with ids, categories, summaries, versions, and maturity status, so you can discover production building blocks before inspect_module, compose_app, or plan_add_module.
inspect_module
no_return_description
Inspect one module contract and return its details, including permissions, hooks, events, resources, storage needs, and customization points. Use this before editing or adding a module so the agent understands the module boundaries.
list_module_docs
no_return_description
List LLM-readable documentation pages and return the available page ids or references for module implementation guidance. Use this to discover local docs before get_module_doc or before changing generated source.
compose_app
no_return_description
Compose a template and selected modules into a checked app contract and lockfile plan without writing files, and return the resulting contract and plan so you can preview the exact app composition before generate_project or deployment planning.
validate_config
no_return_description
Validate a template, module list, and configuration object, and return validation results indicating unsupported modules, missing settings, and invalid config, before generation or deployment. Use this to catch problems before side effects.
generate_project
no_return_description
Generate project source files in memory and return them for agent inspection, including planned source, docs, migrations, and config. This tool does not write files; use it to review the generated output before creating or editing a local project.
run_checks
no_return_description
Run local contract and readiness checks for a template/module composition and return the check results, including module compatibility, generated artifacts, and operational guardrails, before deployment planning.
plan_add_module
no_return_description
Plan an approval-gated module addition against an optional existing microservices.lock.json and return the intended lockfile changes and required review gates without writing files.
check_updates
no_return_description
Check locked module versions against the bundled registry snapshot and return the modules with available updates, so you can identify them before plan_module_upgrade.
plan_module_upgrade
no_return_description
Plan a module version change and return the approval gates, compatibility notes, and lockfile impact, so you can review the upgrade before applying it.
get_secrets_status
no_return_description
Report required secret names and their configured/missing status for a template/module composition without exposing secret values, and return that status report. Use this before deployment planning or remote preview deploys.
create_preview_plan
no_return_description
Create a local preview-deployment readiness plan without mutating remote state and return the plan, including resources, checks, errors, warnings, and required approval steps. Use this before deploy_preview.
deploy_preview
no_return_description
Prepare a remote preview deployment through the microservices.sh control plane and return the resulting preview deployment details, including a deployment id for later status checks. This is a mutating tool and requires confirm: preview; call create_preview_plan first and only proceed after human review.
get_deployment_status
no_return_description
Read a remote preview deployment status from the microservices.sh control plane and return the status, any errors, and deployment metadata. Use this after deploy_preview to poll status or inspect errors.

Selection evidence

Confusable tool pairs.

7 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A Tool B Confidence Why they collide
list_modules list_module_docs medium A task like 'list the modules available' is ambiguous between listing verified module building blocks (list_modules) and listing LLM-readable implementation documentation pages (list_module_docs); both are parameterless list tools sharing 'list' and 'module'.
plan_add_module plan_module_upgrade medium A task like 'plan the payment module change' or 'plan to move to a newer module' is ambiguous between planning an approval-gated addition and planning a version upgrade; both are non-mutating, approval-gated planning tools that take module selectors and lockfiles.
inspect_template inspect_module medium Example ids overlap (e.g. 'booking' appears as an inspect_module example while 'booking-sveltekit' is a template), so a task like 'inspect the booking app' is ambiguous between inspecting a template contract and inspecting a module contract.
list_module_docs get_module_doc medium A task like 'show me the module documentation' is ambiguous between listing available documentation pages and fetching a specific module's doc page; get_module_doc requires a moduleId, and the user may not have specified it yet.
inspect_module get_module_doc medium A task like 'understand the payment module before I edit it' is ambiguous between inspecting the module contract/permissions (inspect_module) and retrieving the LLM-readable implementation guidance (get_module_doc), which is explicitly for coding against a module.
create_preview_plan deploy_preview medium A task like 'create a preview deployment' is ambiguous because create_preview_plan is the documented prerequisite and deploy_preview is the mutating action; an agent may generate the plan instead of deploying, or deploy without the required plan.
run_checks check_updates low A terse task like 'check my modules' is ambiguous between running composition/readiness checks (run_checks) and comparing locked versions against the registry for updates (check_updates), though the descriptions differentiate contract validation from version checking.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard