Public assessment

sint-ai/sint-protocol (sint-mcp)

sint-mcp · v0.1.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 9%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

24.9 / 30

24.9 out of 30

03Economics

19.5 / 20

19.5 out of 20

04Discoverability

12.8 / 20

12.8 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

15 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`sint__status`	name_restates_behavior no_return_description	Return the SINT MCP health snapshot containing the connected servers, current agent identity, approval queue size, and system health, for a quick operational liveness check before admin actions.
`sint__servers`	no_return_description	Return the list of downstream MCP servers with each one's connection status, tool count, and health, to verify the current server topology.
`sint__whoami`	no_return_description	Return the current agent identity as its public key, active token, and session information, to confirm which identity is acting.
`sint__pending`	no_return_description	Return all pending approval requests awaiting human review so they can be approved or denied by ID.
`sint__approve`	no_return_description	Approve the pending escalated action identified by requestId, attributing it to the given approver (defaults to the current agent); returns the approval outcome.
`sint__deny`	no_return_description	Deny the pending escalated action identified by requestId, attributing it to the given denier (defaults to the current agent) with an optional reason; returns the denial outcome.
`sint__audit`	no_return_description	Return up to limit (default 20) most recent decisions and events from the SINT evidence ledger for review.
`sint__add_server`	no_return_description	Dynamically add a downstream MCP server at runtime, spawning it under the unique name with the given command and args; returns confirmation once registered.
`sint__remove_server`	no_return_description	Remove the downstream MCP server identified by name at runtime; returns confirmation of removal.
`sint__revoke_token`	no_return_description	Revoke the active capability token identified by tokenId (admin only), optionally recording a reason; returns confirmation of revocation.
`sint__show_hud`	no_return_description	Update the specified HUD panel (approvals, audit, context, or memory) with the given data and emit an operator.hud.updated event; returns confirmation of the panel update.
`sint__store_memory`	no_return_description	Store the value under a unique key in the memory bank with optional tags and optional persistence beyond the session; returns confirmation of the stored entry.
`sint__notify`	no_return_description	Send a proactive notification to the operator, optionally attaching an action button that invokes an MCP tool; returns confirmation that the notification was sent.
`sint__interface_mode`	no_return_description	Switch the operator interface to the requested display mode (hud, compact, voice-only, or silent); returns confirmation of the mode change.
`sint__revoke_delegation_tree`	no_return_description	Revoke the delegation subtree rooted at rootTokenId, cascade-revoking all descendant tokens, optionally recording a reason; returns confirmation of the revocation.

Selection evidence

Confusable tool pairs.

4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`sint__status`	`sint__interface_status`	medium	Both are status-reporting tools with near-identical names; a vague task like 'what's the current status?' could route to either, since interface_status reports operator/interface state while status reports system state.
`sint__approve`	`sint__deny`	medium	Identical input schemas and both act on a pending escalated action by request ID; a task phrased as 'handle/process/resolve this pending request' does not disambiguate approve vs deny from the definitions alone.
`sint__revoke_token`	`sint__revoke_delegation_tree`	medium	Both revoke a capability token by ID, and delegated sub-agent tokens are also tokens; 'revoke the token I delegated to X' could plausibly pick either revoke_token or revoke_delegation_tree.
`sint__issue_token`	`sint__delegate_to_agent`	medium	Both issue restricted capability tokens with similar inputs (subject/subagentId, resource/toolScope, expiry); a task like 'give this agent a token limited to the filesystem' is ambiguous between them.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard