01Safety
0.0 / 30
What changed in the harness
Selection accuracy 93→100, token cost up 2%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.4 / 30
03Economics
19.8 / 20
04Discoverability
12.1 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
flowzap_validate |
no_return_description |
Check whether the provided FlowZap Code is syntactically valid, intended to be run before creating a playground. Returns a validation result indicating whether the code is valid, including any syntax errors found. |
flowzap_apply_change |
no_return_description |
Apply a structured set of insert/remove/update operations for nodes or edges to the given FlowZap Code, preserving its existing structure rather than regenerating the diagram. Returns the updated FlowZap Code with all patch operations applied. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
flowzap_validate |
flowzap_get_syntax |
medium | A user asking 'check the syntax of my FlowZap code' or 'is this syntax valid' maps to validate, while 'show me the syntax docs' maps to get_syntax. Since get_syntax is the only tool literally about 'syntax' and validate also processes code input, an agent can misroute a syntax-error-fixing request (e.g. 'help me with the syntax of my flow') to get_syntax docs instead of validating the actual code. |
flowzap_diff |
flowzap_apply_change |
medium | Both tools operate on nodes/edges with remove/update vocabulary: diff explains which nodes/edges were added/removed/updated, while apply_change inserts/removes/updates them. A task like 'update my flow by removing node X and showing what changed' is ambiguous between reporting changes (diff) and performing them (apply_change), and neither description strongly distinguishes the singular vs. dual-code input requirement up front. |
flowzap_create_playground |
flowzap_artifact_to_diagram |
medium | A task like 'give me a visual diagram of this app' with raw input (HTTP logs, OpenAPI, code snippet) could route to artifact_to_diagram (converts raw artifacts into visual workflows) or to create_playground (makes a shareable visual diagram), since both yield diagrams and playground is the more generic 'make a diagram' entry point. Confusion is plausible when the input isn't clearly FlowZap Code or clearly a supported artifact type. |
flowzap_create_playground |
flowzap_compliance_check |
low | Both tools return shareable URLs and create_playground's description embeds the compliance coupling ('MUST also call flowzap_compliance_check'), so a compliance/audit request genuinely requires both. A careless agent could call only one (e.g. only the playground for a SOC2 audit request, or only the compliance check without producing the diagram), but the cross-references in both descriptions largely steer a careful agent, so confusion risk is limited. |
Compare the field