Public assessment

chrbailey/promptspeak-mcp-server (@chrbailey/promptspeak-mcp-server)

chrbailey-promptspeak-mcp-server · v0.4.1 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 11%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

20.5 / 30

20.5 out of 30

03Economics

15.5 / 20

15.5 out of 20

04Discoverability

13.3 / 20

13.3 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

47 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`ps_validate_batch`	params_unexplained no_return_description	Validate multiple PromptSpeak frames in a single call, useful for checking delegation chains together. frames is an array of frames, each with the frame text and an optional parentFrame for chain context; validationLevel selects structural, semantic, chain, or full validation depth; strict causes warnings to also fail validation; stopOnFirstError stops validation at the first failing frame. Returns a per-frame validation report listing errors, warnings, and suggestions.
`ps_execute`	params_unexplained no_return_description	Execute a single action under a PromptSpeak frame; the gatekeeper validates the action against the frame's constraints before running it. agentId identifies the executing agent; frame is the governing PromptSpeak frame; action specifies the tool/action to execute, its arguments, and an optional description; parentFrame applies if the call is part of a delegation chain. Returns the gatekeeper's validation verdict and the action's result, or a failure if the constraints are violated.
`ps_execute_dry_run`	params_unexplained	Check whether an action would succeed under a PromptSpeak frame without executing it. agentId identifies the would-be executing agent; frame is the frame the action would be governed by; action is the tool to test and its arguments; parentFrame provides delegation-chain context if applicable. Returns a decision indicating whether the action is allowed and a coverage analysis of how the frame constrains the action.
`ps_execute_batch`	no_return_description	Execute multiple actions under a single PromptSpeak frame, sequentially by default or in parallel when parallel is true; stopOnFirstFailure aborts remaining actions on the first failure (sequential only). Returns per-action validation results and execution outcomes, plus an overall batch summary.
`ps_delegate`	params_unexplained no_return_description	Delegate a task from a parent agent to a child agent, enforcing inheritance rules and constraint propagation between frames. parentAgentId and childAgentId identify the delegating and receiving agents; parentFrame and childFrame are the frames governing each side; task is the delegated work with an optional description, constraints list, and deadline; inheritanceMode selects strict, relaxed, or custom enforcement. Returns the new delegation with its ID and status, or an error if the delegation violates inheritance rules.
`ps_delegate_revoke`	params_unexplained no_return_description	Revoke an active delegation. delegationId identifies the delegation being revoked; parentAgentId identifies the parent agent that created it; reason documents why it is being revoked. Returns the revoked delegation's ID and updated status.
`ps_delegate_list`	params_unexplained no_return_description	List delegations associated with an agent. agentId is the agent to query; role filters to delegations where the agent is the parent, child, or both; status filters to active, completed, revoked, or all delegations. Returns the matching delegations with their IDs, agents, frames, and statuses.
`ps_state_get`	params_unexplained no_return_description	Get the current state of an agent. agentId identifies the agent. Returns that agent's state, including drift metrics and circuit breaker status.
`ps_state_system`	no_return_description	Get the overall system state. Returns a summary of all agents, recent operations, and current drift alerts.
`ps_state_reset`	params_unexplained no_return_description	Reset the state of an agent. agentId identifies the agent; resetCircuitBreaker clears the circuit breaker; resetDriftMetrics clears drift metrics; resetBaseline clears the drift baseline; reason explains why the reset was performed. Returns the agent's state after the reset has been applied.
`ps_state_halt`	params_unexplained no_return_description	Immediately halt an agent by opening its circuit breaker. agentId identifies the agent to halt; reason documents why it is being halted. Returns the agent's updated state, including the now-open circuit breaker.
`ps_state_resume`	params_unexplained no_return_description	Resume a halted agent. agentId identifies the agent to resume; reason documents why it is being resumed; resetMetrics optionally clears accumulated drift metrics on resume. Returns the agent's updated state.
`ps_state_drift_history`	params_unexplained no_return_description	Get the drift history for an agent. agentId identifies the agent; since is a Unix timestamp specifying the earliest history to include; limit caps the number of entries returned. Returns the list of drift records for the agent.
`ps_config_set`	params_unexplained no_return_description	Register a new policy overlay in the configuration. overlayId is a unique identifier for the overlay; overlay is the policy content to register. Returns the registration result, including the overlay ID and whether it is active.
`ps_config_activate`	params_unexplained no_return_description	Activate a previously registered policy overlay. overlayId identifies the overlay to activate. Returns the active overlay's ID and the resulting configuration.
`ps_config_get`	no_return_description	Get the current configuration. Returns the configuration, including the active overlay and current thresholds.
`ps_config_export`	no_return_description	Export the current configuration for backup. Returns a serialized representation of the configuration that can be restored with ps_config_import.
`ps_config_import`	params_unexplained no_return_description	Import a configuration from a backup. data is the serialized configuration string to restore; expectedChecksum is an optional checksum used to verify the import's integrity. Returns the import status and the resulting configuration, or an error if the checksum does not match.
`ps_confidence_set`	params_unexplained no_return_description	Set a confidence threshold; this is the hidden knob for operators. threshold selects which threshold to adjust (parse confidence, coverage confidence, chain confidence, drift threshold, or tripwire threshold); value is the new threshold between 0 and 1. Returns the updated values for all thresholds.
`ps_confidence_get`	no_return_description	Get all confidence thresholds. Returns the current value for every confidence and threshold setting.
`ps_confidence_bulk_set`	params_unexplained no_return_description	Set multiple confidence thresholds in a single call. thresholds is an object mapping threshold names (e.g., parseConfidence, coverageConfidence, chainConfidence, driftThreshold, tripwireThreshold) to their new values. Returns the updated set of all thresholds.
`ps_feature_set`	params_unexplained no_return_description	Set a feature flag. flag is the name of the feature flag to change; enabled turns the feature on or off. Returns the updated state of the flag.
`ps_feature_get`	no_return_description	Get all feature flags. Returns the current state of every feature flag.
`ps_audit_get`	params_unexplained no_return_description	Get audit log entries that match the given filters. Returns the matching audit entries. 'since' is a numeric timestamp that limits results to entries created after it; 'action' limits results to a specific action type; 'limit' caps the number of entries returned.
`ps_expand`	no_return_description	Expand a PromptSpeak expression to natural English. Returns the human-readable English expansion, which safety filters use for human-readable action descriptions.
`ps_registry_lookup`	no_return_description	Resolve a verb symbol to its full definition. Returns the verb's definition, including its aliases.
`ps_registry_propose`	no_return_description	Submit a new verb for review. Returns the created verb record, which is registered with the status 'proposed'.
`ps_registry_status`	no_return_description	Check the lifecycle state and safety classification of a verb. Returns the verb's current lifecycle state and safety classification.
`ps_registry_namespace`	no_return_description	List all verbs registered in a namespace. Returns the list of verbs registered under the given namespace.
`ps_registry_audit`	no_return_description	Get the full change history for a verb. Returns the sequence of registration, transition, and update events for that verb.
`ps_registry_version`	no_return_description	Get the current spec version and registry statistics. Returns the spec version, verb count, and other registry statistics.
`ps_handshake_respond`	no_return_description	Handle an incoming PromptSpeak handshake probe by parsing, validating, and echoing confirmation. Returns the confirmation response to send back to the remote agent.
`ps_capability_get`	no_return_description	Report this server's PromptSpeak capabilities. Returns the capability information, including version, verb count, and namespaces.
`ps_hold_approve`	no_return_description	Approve a held execution request; the operation will then proceed, possibly with the optional modifications supplied. Returns the result of the approval, including whether the operation proceeded.
`ps_hold_reject`	no_return_description	Reject a held execution request; the operation will not proceed. Returns the result of the rejection, including whether the requesting agent was halted.
`ps_hold_config`	no_return_description	Configure hold behavior and thresholds that control when operations are held for human review. Returns the resulting hold configuration: the current config when action=get, or the newly set config when action=set.
`ps_hold_stats`	no_return_description	Get hold statistics and history. Returns hold statistics and the historical decisions, up to 'historyLimit' entries (default: 10).
`ps_security_gate`	no_return_description	Scan code and enforce the security policy: blocks on critical findings, holds high-severity findings for review, warns on medium findings, and logs low/info findings. Returns the gate decision along with the scan findings.
`ps_security_config`	no_return_description	Configure security detection patterns: list patterns, or enable, disable, or change the severity of a pattern. Returns the resulting pattern list or updated configuration.
`ps_symbol_create`	params_unexplained no_return_description	Create a new directive symbol in the registry using a namespaced symbol ID (e.g., Ξ.NVDA.Q3FY25). Returns the created symbol record with its ID and grounding context. The 'how' parameter is an object describing how the work should be executed: 'focus' (what to emphasize), 'constraints' (restrictions/guardrails), and 'output_format' (expected output style).
`ps_symbol_update`	no_return_description	Update an existing symbol, creating a new version with an updated hash. Returns the new symbol version and its updated hash.
`ps_symbol_list`	no_return_description	List symbols with optional filtering by category, tags, creation date, or free-text search. Returns the matching symbols, paged by 'limit' (default: 50) and 'offset'.
`ps_symbol_delete`	no_return_description	Delete a symbol from the registry. Returns the outcome of the deletion, including the deleted symbol ID and reason.
`ps_symbol_import`	no_return_description	Bulk import symbols from external data sources (HuggingFace, JSON, CSV) and register them in the symbol registry. Provide the source format, raw data, a category, and an optional ID prefix and field mappings. The source format must be one of hugface, json, or csv; category must be one of COMPANY, PERSON, EVENT, SECTOR, TASK, KNOWLEDGE, QUERY. Returns a summary of the import, including the number of symbols created and their generated symbol IDs.
`ps_symbol_stats`	no_return_description	Get statistics about the symbol registry, such as totals per category, claim type, or verification status. Returns a summary object containing the requested registry statistics, along with the total number of symbols and any relevant breakdowns.
`ps_symbol_verify`	no_return_description	Record human verification of a symbol claim, upgrading or disputing its status. Set the target symbolId, the new_status (VERIFIED, CORROBORATED, or DISPUTED), the reviewer, and optionally a new_confidence (0-1), supporting evidence sources, and notes. Accusatory claims that lack evidence or have plausible alternative explanations should be DISPUTED. Returns the updated symbol with its new status and confidence.
`ps_symbol_add_alternative`	no_return_description	Add an alternative explanation to a symbol's findings to prevent false positives. When a pattern-based finding could have multiple explanations, document the alternative, its estimated likelihood (0-1), the reasoning, and who added it. Listing high-likelihood alternatives automatically reduces confidence in the original claim. Returns the updated symbol showing the added alternative and the adjusted confidence.

Selection evidence

Confusable tool pairs.

8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`ps_confidence_set`	`ps_confidence_bulk_set`	high	Both set confidence thresholds with overlapping threshold/value concepts; a task like "set chain confidence to 0.8" maps equally to the single or bulk tool, and bulk_set's schema accepts per-threshold values, so an agent may pick the wrong one.
`ps_execute`	`ps_execute_batch`	medium	A task to "run these actions under this frame" is ambiguous between executing one action and executing many; an agent could pick the single-action tool for a multi-action request or vice versa since both share execute/frame/action vocabulary.
`ps_delegate`	`ps_delegate_list`	low	The verbs (delegate vs list) are distinct, but a task like "show me my delegations" is clearly list while "delegate this" is clearly delegate; confusion only arises for loosely-worded tasks about the delegation module, so plausibility is low.
`ps_audit_get`	`ps_registry_audit`	medium	Both surface audit/history information; a task like "get the audit trail for verb ::analyze" could select the generic audit_get (which takes an action filter and returns audit log entries) instead of the verb-specific registry_audit, or vice versa.
`ps_hold_approve`	`ps_hold_reject`	medium	Both act on the same held-request object with matching holdId input; a task like "handle this pending hold" or "resolve the held execution" is genuinely ambiguous about whether to approve or reject until polarity is specified explicitly.
`ps_config_set`	`ps_config_activate`	low	Both concern policy overlays: register a new overlay vs activate a registered one. A task like "turn on the policy overlay" could name register when activate is needed, but the wording is specific enough that confusion is only marginally plausible.
`ps_config_export`	`ps_config_import`	medium	Reverse-direction operations sharing config/backup vocabulary; a task like "restore the saved configuration" or "apply the backup" could select the wrong direction, especially since export's description explicitly mentions backup, pulling agents toward the wrong member.
`ps_registry_lookup`	`ps_registry_status`	medium	Both take the same verb symbol and return registry information about it; a task like "check the state of verb ::analyze" could be mapped to lookup (full definition) rather than status (lifecycle/safety), since first-line descriptions blur definitions with lifecycle state.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard