01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100→98, token cost down 0%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.1 / 30
03Economics
13.8 / 20
04Discoverability
17.5 / 20
Highest-impact fix
Estimated gain +6 pointsReduce the tool-definition and task token payload
Consolidate overlapping tools and trim repetitive schema text while preserving behavior, parameter, and return semantics.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
get_drug_rules |
no_return_description |
Look up Taiwan NHI drug payment rules (藥品給付規定) and return the matching rule records — each record states the drug class / condition it covers, whether that class is or is not reimbursable and under what condition, the severity rating, and the source citation from the official 健保藥品給付規定. Filter by `specialty`, `rejection_code`, or `drug_category_query` (at least one required); the returned set is a curated subset, not exhaustive. **Use when** an agent is checking whether a drug choice will trigger a rejection before submission, or wants to read the rule behind a known rejection code. **Don't use** to identify a specific drug (brand / generic / strength) — call `lookup_drug` instead; for the underlying audit clauses, use `lookup_audit_clauses_for_procedure` or `lookup_audit_clauses_for_specialty`. **Reference only** — the official 健保藥品給付規定 is authoritative. Curated by OPDSTAR (https://opdstar.com). |
Selection evidence
Confusable tool pairs.
12 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
lookup_audit_clauses_for_procedure |
lookup_audit_clauses_for_specialty |
medium | Both return 審查注意事項 audit-clause previews and each description cross-references the other. A task like 'check the audit clauses for this dermatology claim' where the user holds both a procedure code and a specialty offers no surface signal about which input keys the lookup, so the agent could pick the specialty-browse tool instead of the procedure-keyed tool (or vice versa). |
lookup_rejection_code |
lookup_fee_code |
medium | Both are single-code lookups returning NHI code explanations, and the shared 'lookup + code' tokens do not reveal domain. A task like 'look up what this NHI code means / why is code 00101B flagged' could route a fee-schedule code to lookup_rejection_code or a rejection code to lookup_fee_code; only the format rule (NNNNL rejection vs 5-8 char fee) disambiguates. |
search_nhi_wiki |
search_nhi_interpretations |
high | Both are natural-language searches over official 健保署 knowledge bases and, notably, both descriptions cite the same example topic (慢性病連續處方箋天數上限 vs continuous-prescription day limit interpretation). A task like 'search NHI rules about continuous-prescription day limits' has no signal for choosing the general wiki over the binding 行政函釋 corpus. |
lookup_audit_clauses_for_procedure |
lookup_audit_indicator |
medium | lookup_audit_indicator also accepts a procedure_code and both concern audit/review risk. A task like 'check the audit/indicator rules for procedure 23401C' is ambiguous between clause-level caveats (amount/frequency/indication) and percentage-cap audit indicators. |
lookup_audit_clauses_for_specialty |
lookup_audit_indicator |
medium | lookup_audit_indicator filters by specialty and both tools answer 'what audit rules apply to my claims.' A task like 'what audit rules/indicators apply to ophthalmology' could route to either the specialty clause browser or the percentage-cap indicator browse. |
lookup_rejection_code |
count_appeal_precedents_for_rejection_code |
medium | Both accept a rejection_code and overlap in purpose. A task like 'tell me about rejection 0312A / should we contest it' is ambiguous between the human-readable explanation and the dispute win-rate signal; lookup_rejection_code even lists count_appeal_precedents as a typical follow-up. |
get_drug_rules |
lookup_drug |
medium | A drug-name task like 'what are the NHI payment rules for augmentin' is ambiguous: lookup_drug matches by name but only identifies the drug, while get_drug_rules holds the rules but filters by drug category/rejection_code/specialty rather than name, so an agent can plausibly select either to answer a drug-rules question. |
get_indicator |
lookup_audit_indicator |
high | Both tools are literally 'indicator' lookups over two different NHI indicator systems (prescribing-pattern vs percentage-cap audit), and numeric/P-prefixed indicator codes overlap in format. A task like 'look up indicator 001 / which indicator fired on this claim' gives no systemic signal without reading the fine print of both descriptions. |
search_nhi_wiki |
search_taiwan_drug |
medium | A task like 'search NHI for drug augmentin / is it covered' could route to search_taiwan_drug (registry record, billing code) or to search_nhi_wiki's drugs category (formulary/payment-limit excerpts); both are search tools over NHI drug information. |
get_drug_rules |
search_taiwan_drug |
medium | search_taiwan_drug's typical follow-up explicitly chains to get_drug_rules, so for a drug-name question like 'check the NHI rules for augmentin' an agent may pick search_taiwan_drug (which can locate the drug by name) over get_drug_rules (which holds the rules but matches by category, not name) — or the reverse. |
search_nhi_interpretations |
search_audit_guidelines |
medium | Both are free-text searches over official NHI rule documents using the same query style (Chinese/English, e.g. rule topics like chronic-prescription limits). A task like 'search NHI rules about 慢性病處方' is ambiguous between 行政函釋 rulings and 審查注意事項 audit clauses. |
search_nhi_wiki |
search_audit_guidelines |
medium | search_nhi_wiki includes an explicit 'audit' category covering review/rejection rules, overlapping search_audit_guidelines' dedicated 審查注意事項 search. A task like 'search NHI for audit/review rules on 抗生素' has overlapping scope and could route to either. |
Compare the field