0.0 / 30
What changed in the harness
Selection accuracy 98→100, token cost up 4%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
01Safety
02Legibility
19.2 / 30
03Economics
13.3 / 20
04Discoverability
12.2 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
26 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
ssh_upload |
params_unexplained no_return_description |
Uploads one local file to a remote destination path over SFTP on the named server (the server config entry to connect to), overwriting any existing remote file at that path. Mutates remote state and is not idempotent beyond replacing the target. Creates no backup. Requires the local file to exist. Does not use sudo, so the remote path must be writable by the configured SSH user. Blocked entirely on servers set to readonly or restricted security mode. Returns success status and the remote path written. For directory trees use ssh_sync instead. |
ssh_download |
params_unexplained no_return_description |
Downloads one remote file from the named server (the server config entry to connect to) to a local destination path over SFTP, overwriting any existing local file at that path. Affects only the local filesystem and is read-only on the remote side, so it stays allowed even on servers in readonly or restricted security mode. Reads the remote file using the configured SSH user, which must have permission to read it. Returns success status and the local path written. Handles single files only; use ssh_sync for directories. |
ssh_sync |
no_return_description |
Synchronizes files or directories between local and remote using rsync over SSH on the named server. Each of source and destination must carry a local: or remote: prefix and one side must be local and the other remote; with no prefix it assumes a push from local to remote. Mutates the destination. Setting delete true removes destination files absent from source, which is destructive; dryRun true previews without changing anything. Compression is on by default. Password authentication requires sshpass installed locally. Blocked on readonly or restricted servers. Timeout defaults to 30000 ms. Returns the rsync command output including transfer summary and exit status. |
ssh_session_send |
no_return_description |
Runs one command inside an already-open session identified by its session ID, reusing the persisted working directory, environment, and history of that shell. Mutates remote state like any shell command and is not idempotent; cd and export update the saved context for subsequent calls. Commands run through a bash-style shell (Unix-oriented). The security policy of the underlying server is enforced, so readonly or restricted servers may refuse. Default timeout is 30000 ms. Returns the command's stdout, stderr, and exit code. |
ssh_session_close |
no_return_description |
Terminates an open SSH session given its session ID, writing exit to the remote shell, ending it, and discarding its in-memory history and context; the session ID becomes unusable afterward. Destructive to session state but does not delete remote files. Passing the literal value all closes every active session at once, ignoring individual close errors. It does not drop the pooled underlying connection, only the interactive shell. Returns the session ID(s) closed and, for "all", a count closed. |
ssh_group_manage |
params_unexplained no_return_description |
Creates, updates, deletes, and inspects named server groups used by ssh_execute_group, persisting changes to local configuration only with no remote side effects. The action selects the operation: create, update, delete, add-servers, remove-servers, or list. Every action except list requires name; add-servers and remove-servers also require servers, the array of server names to add to or remove from the group. Optional strategy, delay, and stopOnError set default group execution behavior. Returns the resulting group definition (or, for list, the array of all groups). |
ssh_deploy |
no_return_description |
Deploys a list of local files to remote paths on the named server, uploading each to a temporary location first and then moving it into place. Mutates remote state. By default it backs up any existing target file before overwriting; backup can be disabled per call. Options can set owner and permissions, supply a sudo password, and name a single service to restart afterward. Detects sensible owner and permission defaults from the remote path. Runs pre and post deploy hooks. Blocked entirely on servers in readonly or restricted security mode. Returns per-file deployment results, backup paths created, and the restart outcome if a service was specified. |
ssh_hooks |
no_return_description |
Manages automation hooks that fire around SSH operations such as pre-deploy, toggling them on or off in local configuration only with no immediate remote action. The action selects behavior: list shows each hook with its enabled state, description, and action count; enable and disable flip a hook and both require the hook name; status summarizes which hooks are currently enabled versus disabled. Toggling persists and affects later operations. Returns the hook list, updated hook state, or status summary depending on action. |
ssh_profile |
no_return_description |
Manages SSH Manager profiles that bundle command aliases and hooks for different project types, affecting local configuration only with no remote side effects. The action selects behavior: list shows available profiles and the active one, current shows the active profile details, and switch activates a named profile and requires the profile argument. Returns the profile list, current profile details, or switch confirmation; a successful switch also reports that Claude Code must be restarted before the new profile takes effect. |
ssh_connection_status |
no_return_description |
Inspects and manages the pooled SSH connections held by this server process; affects only local in-memory connections, never remote state. The action parameter selects: status lists active connections with age and keepalive (read-only); reconnect closes then reopens one connection; disconnect closes one connection; cleanup drops aged-out and dead connections. The server parameter is required for reconnect and disconnect and ignored otherwise. Returns the connection list for status, or a confirmation of the action taken for reconnect/disconnect/cleanup. |
ssh_tunnel_create |
params_unexplained |
Opens a new SSH connection to the named server and starts a port-forwarding or SOCKS proxy tunnel that keeps running until closed. The type parameter selects local forward, remote forward, or dynamic SOCKS5 proxy. localPort, the local TCP port to bind the tunnel to, is always required; remoteHost and remotePort are required for local and remote types but ignored for dynamic. localHost defaults to 127.0.0.1. Returns a tunnel ID used later to close it. |
ssh_tunnel_close |
no_return_description |
Tears down active SSH tunnels created earlier, freeing the bound local ports; this affects only local tunnel state, not the remote host. Exactly one of tunnelId or server must be supplied: tunnelId closes that single tunnel, while server closes every tunnel for the named server and reports how many were closed. Supplying neither raises an error. Closing is final and cannot be undone. Returns confirmation of the tunnel(s) closed, or the closed count when server is used. |
ssh_key_manage |
no_return_description |
Manages SSH host key fingerprints in your local known_hosts file for the named server. The action parameter selects: verify, check, and list are read-only comparisons or listings; accept adds or updates the host key in known_hosts; remove deletes it. accept and remove mutate local state and are blocked on servers configured as readonly. server is required for every action except list. autoAccept defaults to false and should be used with caution. Returns the fingerprint/verification result, updated known_hosts entry, or the alias list depending on action. |
ssh_alias |
no_return_description |
Manages local name aliases that let you reference a configured server by a shorter or alternative name. The action parameter selects add, remove, or list. add creates an alias pointing to an existing server and requires both alias and server; remove deletes an alias and requires alias; list shows all aliases (read-only). add and remove persist the alias mapping locally. The target server must already exist for add to succeed. Returns the resulting alias mapping, or the full alias list for list. |
ssh_backup_create |
params_unexplained no_return_description |
Creates a database or file backup on the remote server (the server, identified by its config name, to connect to and run the backup on) over SSH, writing a compressed archive plus a JSON metadata file into backupDir. Supports mysql, postgresql, mongodb, and files (full is not yet implemented and errors). Database types require database; files requires paths. After writing it prunes backups older than retention days (default 7); compress defaults to true. Runs pre-backup and post-backup hooks. Returns the created backup's id, path, size, and metadata. |
ssh_backup_list |
params_unexplained |
Lists existing backups found in backupDir on the remote server (the server, identified by its config name, whose backup directory is inspected), returning each backup id, type, database or paths, size, compression, retention, status, and creation time parsed from stored metadata. Read-only: it inspects the filesystem and mutates nothing. Optional type filters results to mysql, postgresql, mongodb, files, or full. backupDir defaults to the configured backup directory. |
ssh_backup_restore |
params_unexplained no_return_description |
Restores a previously created backup identified by backupId on the named server (the server, identified by its config name, to connect to and run the restore on), reading its metadata to pick the engine. This is destructive and overwrites the target: PostgreSQL runs pg_restore with --clean --if-exists which DROPs existing objects, MongoDB runs mongorestore --drop, and MySQL pipes the dump into the live database replacing matching objects. Supports mysql, postgresql, mongodb, and files. Runs pre-restore and post-restore hooks. Returns the restore outcome including status and any output/errors from the restore command. |
ssh_backup_schedule |
params_unexplained no_return_description |
Schedules a recurring backup on the remote server (the server, identified by its config name, to connect to and install the cron job on) by writing an executable bash script to /usr/local/bin/ssh-manager-backup-NAME.sh and installing a crontab entry for the given cron expression. Mutates the remote filesystem and crontab, and typically needs root to write that path. Supports mysql, postgresql, mongodb, and files; the generated script also deletes backups older than retention days (default 7). Returns the installed script path and crontab entry. |
ssh_health_check |
params_unexplained |
Runs a comprehensive read-only health check on the named server (the server, identified by its config name, to connect to and inspect) by executing diagnostic shell commands over SSH, then returns parsed JSON with overall status, CPU, memory, disk usage, and uptime. It only reads metrics and changes nothing on the remote host. Set detailed to true to additionally include load average and network metrics; it defaults to false. Critical CPU, memory, or disk conditions are surfaced in a critical_issues list. |
ssh_service_status |
params_unexplained |
Checks the running state of the named system services on a remote server (the server, identified by its config name, to connect to) by querying each one over SSH, returning JSON per service plus running and stopped counts and an aggregate health rating. Read-only: it inspects status without starting, stopping, or restarting anything. The services array parameter is required and lists the service names to check, for example nginx, mysql, or docker; common names are resolved to their actual unit names automatically. |
ssh_process_manager |
params_unexplained |
Lists, inspects, or terminates processes on a remote server (the server, identified by its config name, to connect to) over SSH. The action parameter selects: list returns top processes (read-only), info returns details for one process (read-only), and kill sends a signal to terminate a process and mutates remote state. pid is required for kill and info. kill is blocked on servers configured as readonly. signal defaults to TERM, sortBy defaults to cpu, and limit defaults to 20; filter narrows the list by name or command. |
ssh_alert_setup |
params_unexplained no_return_description |
Configures and evaluates CPU, memory, and disk usage alert thresholds for a remote server (the server, identified by its config name, to connect to). The action parameter selects: set writes the threshold config to /etc/ssh-manager-alerts.json on the remote host (mutating, may need write access to /etc, and is blocked on readonly servers); get reads back that config; check reads current metrics and compares them to stored thresholds. get and check are read-only. enabled defaults to true; check errors if no config exists yet. Returns the stored/updated threshold config for set and get, or the current metrics with any breached thresholds for check. |
ssh_db_dump |
params_unexplained |
Dumps a database to a file on the remote server (the server, identified by its config name, to connect to) over SSH; it reads data only and does not modify the database. database names the database to dump. Supports mysql (using --single-transaction --routines --triggers), postgresql (custom format with --clean --if-exists, restorable via pg_restore), and mongodb. compress defaults to true and gzips the output. The optional tables list applies to MySQL and PostgreSQL only and is ignored for MongoDB. |
ssh_db_import |
params_unexplained no_return_description |
Imports a dump file into a target database on the remote server (the server, identified by its config name, to connect to) and is destructive to existing data. PostgreSQL uses pg_restore --clean --if-exists which DROPs existing objects before loading; MongoDB uses mongorestore with --drop controlled by the drop flag (default true); MySQL pipes the file into the live database, replacing objects defined in it. Supports mysql, postgresql, mongodb. Compressed .gz inputs are decompressed automatically. Returns the import command's output and exit status. |
ssh_db_list |
params_unexplained |
Lists database objects on the remote server (the server, identified by its config name, to connect to) for the given engine without modifying anything. When database is provided it lists the tables (SQL) or collections (MongoDB) of that database; when omitted it lists all databases with common system databases filtered out. Supports mysql, postgresql, and mongodb. Returns the items and a count. Read-only and safe to call repeatedly. |
ssh_db_query |
params_unexplained |
Runs a read-only query against a remote database on the named server (the server, identified by its config name, to connect to) and database (the target database name). For mysql and postgresql it is strictly limited to SELECT: the query must begin with SELECT and any insert, update, delete, drop, create, alter, truncate, grant, revoke, or exec keyword is rejected before execution. For mongodb it runs a find() and requires the collection parameter. Returns the raw command output as text. |
Selection evidence
Confusable tool pairs.
12 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
ssh_command_alias |
ssh_alias |
high | Both tools manage something called an 'alias' with add/remove/list actions; a request like 'add an alias for prod' doesn't specify whether it should map to a server name (ssh_alias) or a full shell command (ssh_command_alias), so the agent could easily pick the wrong one. |
ssh_execute |
ssh_execute_sudo |
medium | A task like 'run this command as root/with elevated privileges' could be satisfied either by ssh_execute_sudo or by ssh_execute with 'sudo' typed into the command string, so the agent may default to the wrong tool for privilege escalation. |
ssh_execute |
ssh_execute_group |
medium | If the task says 'run this on the production servers' without clarifying whether 'production' is a single server name or a configured group, the agent could call ssh_execute repeatedly instead of the group-aware ssh_execute_group, or vice versa. |
ssh_backup_create |
ssh_backup_schedule |
medium | A vague request like 'set up daily backups of the database' is ambiguous between creating one backup now (ssh_backup_create) and installing a recurring cron-based backup job (ssh_backup_schedule). |
ssh_connection_status |
ssh_service_status |
medium | 'Check the status of the server' is ambiguous between inspecting the local pooled SSH connection (ssh_connection_status) and checking whether a named system service like nginx is running (ssh_service_status). |
ssh_session_list |
ssh_tunnel_list |
medium | A request to 'show active connections to the server' could plausibly mean open interactive shell sessions or open port-forwarding tunnels, and both tools return similar-looking active/state/id metadata. |
ssh_session_start |
ssh_session_send |
medium | For 'run this command in a persistent shell', an agent unsure whether a session already exists might call ssh_session_start again instead of reusing an existing session with ssh_session_send, especially in multi-turn tasks. |
ssh_session_close |
ssh_tunnel_close |
medium | 'Close this connection/id' is ambiguous between terminating an interactive shell session and tearing down a port-forwarding tunnel, both identified by an opaque id string. |
ssh_upload |
ssh_download |
medium | Directional wording like 'transfer the file between local and server' or 'sync this file to/from the server' can be misparsed, causing the agent to invoke the SFTP transfer in the wrong direction. |
ssh_hooks |
ssh_profile |
low | Both manage local automation-related configuration bundles (hooks vs profiles that bundle hooks/aliases); a task like 'enable the deploy hooks for this profile' could momentarily point at the wrong management tool. |
ssh_execute_group |
ssh_group_manage |
low | A task like 'add these servers to the production group and run this command' involves both tools in sequence, and an agent could mistakenly call ssh_execute_group when it meant to first define/update the group via ssh_group_manage. |
ssh_backup_list |
ssh_db_list |
low | A phrase like 'list what backups exist for the database' could be misrouted to ssh_db_list (which lists databases/tables) instead of ssh_backup_list (which lists stored backup archives). |
Compare the field