01Safety
0.0 / 30
What changed in the harness
Selection accuracy 43→50, token cost down 0%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
18.2 / 30
03Economics
15.6 / 20
04Discoverability
7.5 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
flowzap_validate |
no_return_description |
Validates FlowZap Code syntax and returns whether the code is valid, along with a list of any syntax errors found (e.g., message, line, and column). Use this to check FlowZap Code before creating a playground. |
flowzap_get_syntax |
no_return_description |
Returns FlowZap Code syntax reference documentation and example snippets covering nodes, edges, lanes, and shapes, to help you learn how to write valid FlowZap Code for workflow diagrams. |
flowzap_apply_change |
no_return_description |
Applies a list of structured patch operations (insert/remove/update nodes or edges) to existing FlowZap Code and returns the updated FlowZap Code. Safer than regenerating entire diagrams, since it preserves existing structure. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
flowzap_validate |
flowzap_get_syntax |
medium | A request like 'check my FlowZap syntax' or 'is this the right FlowZap syntax' is ambiguous between validating a specific code snippet (flowzap_validate) and fetching general syntax documentation/examples (flowzap_get_syntax). |
flowzap_create_playground |
flowzap_artifact_to_diagram |
medium | When given raw artifacts (HTTP logs, OpenAPI spec, code snippets) and asked to 'create a diagram' or 'visualize this', an agent could call flowzap_create_playground directly even though it requires FlowZap Code input, instead of first converting the artifact via flowzap_artifact_to_diagram — both tools are framed around producing a visual diagram/URL. |
flowzap_create_playground |
flowzap_compliance_check |
medium | For a task like 'audit my architecture for SOC2/GDPR compliance', both tools can produce a diagram and shareable URL, and their descriptions explicitly cross-reference each other as required companions — an agent could plausibly call only flowzap_create_playground (skipping the audit) or only flowzap_compliance_check (assuming it alone suffices) instead of invoking both as intended. |
Compare the field