01Safety
0.0 / 30
Public leaderboard
Public assessment
localstack/localstack-mcp-server (@localstack/localstack-mcp-server)
What changed in the harness
Selection accuracy 98→100, token cost up 5%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
24.6 / 30
03Economics
17.5 / 20
04Discoverability
16.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
14 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
localstack-app-inspector |
no_return_description |
Query and manage App Inspector traces, spans, and events to review deployed LocalStack applications. Returns the requested status, trace/span/event data, or a list with a pagination token, depending on the action. |
localstack-aws-client |
no_return_description |
Executes an AWS CLI command against the running LocalStack container using the 'awslocal' wrapper. Returns the command's combined stdout/stderr output and exit status. |
localstack-aws-replicator |
no_return_description |
Replicate external AWS resources into a running LocalStack instance using the AWS Replicator HTTP API. Returns the created job's id and status, a job's current status, a list of jobs, or the list of supported resource types, depending on the action. |
localstack-chaos-injector |
no_return_description |
Injects, manages, and clears chaos faults and network effects in LocalStack to test system resilience. Returns the active fault rules or latency setting after the change, or the current configuration for get actions. |
localstack-cloud-pods |
no_return_description |
Manages remote LocalStack Cloud Pods with following actions: save, load, delete. Returns a confirmation of the save, load, or delete operation on the named pod. |
localstack-deployer |
no_return_description |
Deploys or destroys AWS infrastructure on LocalStack using CDK, Terraform, or SAM. Returns the deploy/destroy/create-stack/delete-stack command output, including resource outputs or errors. |
localstack-docs |
no_return_description |
Search the LocalStack documentation to find guides, API references, and configuration details. Returns a list of matching documentation entries with titles, URLs, and excerpts, up to the given limit. |
localstack-ephemeral-instances |
no_return_description |
Manage cloud-hosted LocalStack Ephemeral Instances: create, list, fetch logs, and delete. Returns the created/target instance's details, the list of instances, its log output, or a deletion confirmation, depending on the action. |
localstack-extensions |
no_return_description |
Install, uninstall, list, and discover LocalStack Extensions from the marketplace. Returns the list of installed or available extensions, or a confirmation of the install/uninstall operation. |
localstack-iam-policy-analyzer |
no_return_description |
Configures LocalStack's IAM enforcement and analyzes logs to automatically generate missing IAM policies. Returns the current or newly set enforcement mode, or the generated IAM policy document from log analysis. |
localstack-logs-analysis |
no_return_description |
LocalStack log analyzer that helps developers quickly diagnose issues and understand their LocalStack interactions. Returns a summary, an error list, a request list, or raw log lines, depending on the selected analysisType. |
localstack-management |
no_return_description |
Manage LocalStack lifecycle: start, stop, restart, or check status. Returns the resulting status of the specified LocalStack stack/service after the action completes. |
localstack-snowflake-client |
no_return_description |
Execute SQL queries and commands against the LocalStack Snowflake emulator using the Snowflake CLI (snow). Use this to run SELECT queries, DDL (CREATE/DROP), DML (INSERT/UPDATE/DELETE), SHOW DATABASES/SCHEMAS/TABLES, DESCRIBE TABLE, and any other Snowflake SQL. Returns the query result rows and output for execute, or the connection status for check-connection. |
localstack-state-management |
no_return_description |
Export, import, reset, and inspect LocalStack state using local file-based workflows on disk. Returns a confirmation of the export/import/reset operation (including the file path used), or the inspected state details. |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
localstack-management |
localstack-state-management |
high | Both tools manage the overall condition of a running LocalStack instance and their names differ by only one word. A request like 'reset LocalStack' or 'clear out LocalStack' is ambiguous between restarting the container (management's restart/stop actions) and wiping persisted state via the State REST API (state-management's reset action), so an agent could easily pick the wrong one. |
localstack-deployer |
localstack-management |
medium | localstack-deployer has create-stack/delete-stack actions for CloudFormation stacks, while localstack-management's 'service' parameter is also documented as 'the LocalStack stack/service to manage' (aws or snowflake). A vague instruction like 'delete the stack' or 'stop the stack' could be misrouted to the container-lifecycle tool instead of the CloudFormation stack tool, or vice versa. |
localstack-aws-client |
localstack-aws-replicator |
medium | Both operate on AWS resources inside LocalStack. A task like 'bring my production S3 bucket into LocalStack' could plausibly be attempted either by scripting it via raw awslocal CLI commands (aws-client) or via the dedicated replication job API (aws-replicator), especially since the agent may not immediately recognize the replicator is the purpose-built path. |
localstack-aws-client |
localstack-snowflake-client |
medium | Both tools are framed as 'execute a command/query against LocalStack using a CLI' with a single 'action'/'command' style interface. If a task mentions running a query or command against 'LocalStack' without specifying AWS vs Snowflake emulator (e.g. 'run this query on my local stack'), an agent could select the wrong client. |
localstack-cloud-pods |
localstack-ephemeral-instances |
medium | Both manage cloud-hosted LocalStack state/instances and ephemeral-instances' create action even accepts a cloudPod parameter to seed state from a Cloud Pod. A request like 'spin up a cloud copy of my LocalStack state' is ambiguous between saving/loading a Cloud Pod snapshot and creating a new ephemeral instance initialized from one. |
Compare the field