01Safety
0.0 / 30
What changed in the harness
Selection accuracy 92→83, token cost up 2%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.8 / 30
03Economics
18.1 / 20
04Discoverability
11.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
authenticate |
no_return_description |
ReportFlow への OAuth2 認証を行います。ブラウザが起動し、ログイン・ワークスペース選択・consent を経て、発行されたトークンを keychain（または XDG file）に保存します。認証が成功したかどうか（成功/失敗）と、保存されたトークンの保存先を返します。他のツールが認証エラーを返したら、まずこのツールを呼んでください。force=true を指定すると、既存トークンを破棄して再認証します。 |
get_design_parameters |
no_return_description |
デザインテンプレートのパラメータ構造を返します。UUID 形式の designId と version でデザインを指定すると、帳票生成に必要なパラメータの型・構造を取得できます。返されたパラメータ構造は、generate_pdf_sync 等の PDF 生成ツールに渡す params を組み立てる際に参照します。version を省略すると最新版のパラメータ構造を取得します。 |
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
generate_pdf_async |
generate_pdfs_async |
medium | Both are async and return requestId/files immediately; natural phrasing like 'PDFを生成して' may not state a file count, and the singular/plural (content vs contents) distinction is only signaled in descriptions, so an agent can mis-select for a single/multi-file task. |
generate_pdf_sync |
generate_pdf_async |
medium | A user rarely says '同期'/'非同期'; 'PDFを生成してダウンロードして' could map to either sync (returns URL in one call) or async (requires download_file). The behavior difference is explained in descriptions but the agent must infer execution mode from vague intent. |
generate_pdf_sync |
generate_pdfs_sync |
medium | Both sync tools exist; the only differentiator is single PDF vs ZIP of multiple PDFs. A task like 'この請求書のPDFを生成して' when multiple invoices are implied is ambiguous on count, risking selection of the wrong arity despite clear descriptions. |
generate_pdfs_sync |
generate_pdfs_async |
medium | 'まとめてPDFを生成して' is ambiguous about whether the agent should block for a local ZIP path (sync) or just initiate and return a requestId (async) requiring download_zip later; sync/async intent is not usually expressed in natural language. |
download_file |
download_zip |
low | A phrase like '生成したファイルをダウンロードして' doesn't specify file vs zip; however each tool's description explicitly ties it to its generator (requestId+fileId vs requestId), and the previously chosen async tool disambiguates, so confusion is only mildly plausible. |
Compare the field