01Safety
0.0 / 30
What changed in the harness
Selection accuracy 86→69, token cost down 0%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
24.0 / 30
03Economics
7.9 / 20
04Discoverability
12.6 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
link_decision |
params_unexplained |
Attach an EXISTING decision (from record_decision) to one more entity — a service, file, symbol, contract, or domain — with a typed relationship: "motivates" (decision caused this entity to exist), "constrains" (decision limits how it can evolve), "documents" (decision explains it), "implements" (entity is the concrete realization of the decision). entity_id must be the ID of that specific entity matching entity_type — a canvas service_id, CodeFile id, CodeSymbol id, contract_id, or domain_id. Use for post-hoc linking: e.g. a month after recording an ADR you realize it also motivates a new service. Unlike record_decision (which can include initial links via the links[] array in one call), link_decision adds ONE link at a time to an already-persisted decision. Unlike get_decisions_for (read), this is a write. Prerequisites: Tentra API auth + existing decision_id + valid entity_id of the chosen entity_type. Write path. Response: { ok: true, link_id }. |
Selection evidence
Confusable tool pairs.
11 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
analyze_codebase |
explain_codebase |
high | A request like 'analyze/explain my codebase' is nearly synonymous in natural language, but analyze_codebase scans a local repo from scratch to create a NEW architecture diagram (write, needs filesystem), while explain_codebase narrates an ALREADY-INDEXED code graph (read-only, needs prior index_code). An agent could pick the wrong one based on phrasing alone. |
lint_architecture |
sync_architecture |
high | Both answer 'is this design OK / what's wrong with it?' — lint_architecture checks the saved diagram in isolation (no codebase needed) while sync_architecture diffs the diagram against real code (needs codebasePath). A vague validation request could trigger either. |
update_architecture |
sync_architecture |
medium | A task like 'sync my architecture with the current code' could plausibly be routed to sync_architecture (read-only drift report) or mistakenly to update_architecture (which requires the agent to already have new services/connections data to apply) since both deal with reconciling architecture state to reality. |
set_service_mapping |
set_domain_membership |
high | Both 'tag' a file/symbol with an identifier via a 'set_*' call. A request like 'assign this file to payments' is ambiguous between mapping it to a concrete canvas service (set_service_mapping) versus tagging it with an abstract business domain (set_domain_membership). |
record_contract |
bind_contract |
medium | Both are contract write operations invoked with similar phrasing ('add/attach this contract to X'). record_contract creates the contract entity itself while bind_contract links an existing contract to a code symbol — an agent might call bind_contract before the contract exists, or record_contract when it should link a symbol. |
record_decision |
link_decision |
medium | A phrase like 'this decision also affects service X' could be handled via record_decision's links[] array (if creating) or link_decision (if the decision already exists) — an agent unsure whether the decision was previously recorded could pick the wrong one. |
index_code |
index_code_continue |
medium | Both are part of the same indexing workflow with very similar names; a request like 'continue indexing' or 'keep going' after a partial index could be mis-routed to index_code (which always starts a fresh walk/snapshot) instead of index_code_continue (which advances an existing job_id), especially if the agent doesn't retain the job_id context. |
get_architecture |
export_architecture |
medium | A request like 'give me the architecture' or 'show me the diagram' is ambiguous between get_architecture (returns raw JSON graph) and export_architecture with format=mermaid/markdown (returns a rendered/shareable representation). |
query_symbols |
get_symbol_neighbors |
medium | Both explore the code graph for a symbol's relationships. A task like 'find code related to X' could be handled by query_symbols (name-based search) or get_symbol_neighbors (graph traversal from a known symbol_id) depending on whether the agent already resolved an ID, leading to inconsistent tool choice. |
record_semantic_node |
record_embedding |
low | Both persist per-file/symbol metadata during indexing ('record_' write calls tied to snapshot_id/job_id). A vague instruction to 'store info about this file' could be misapplied to record_embedding (vector-based) instead of record_semantic_node (human-readable purpose), though the vector requirement usually disambiguates. |
explain_codebase |
explain_code_path |
low | Both start with 'explain_' and relate to understanding code, but scope differs sharply (whole-repo narrative vs shortest path between two specific symbols); a loosely worded 'explain how this works' request could momentarily be misrouted before the agent notices the missing two symbol_ids required by explain_code_path. |
Compare the field