Public assessment

tatsuju/opdstar-nhi-mcp (@opdstar/nhi-mcp)

opdstar-nhi-mcp · v0.9.2 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 95→95, token cost up 1%, unconfirmed writes 100%→100%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

30.0 / 30

30.0 out of 30

02Legibility

25.8 / 30

25.8 out of 30

03Economics

13.7 / 20

13.7 out of 20

04Discoverability

18.2 / 20

18.2 out of 20

Highest-impact fix

Estimated gain +6 points

Reduce the tool-definition and task token payload

Consolidate overlapping tools and trim repetitive schema text while preserving behavior, parameter, and return semantics.

Description evidence

Defects and rewrites.

2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`get_drug_rules`	no_return_description	Look up Taiwan NHI drug payment rules (藥品給付規定). Filter by `specialty`, `rejection_code`, or `drug_category_query` (at least one required). Returns a list of matching rule entries — drug_category, applicable specialty, reimbursement status (when a drug class is / isn't payable), severity, and source citation — or an empty list if no rules match the given filters. Use when an agent is checking whether a drug choice will trigger a rejection before submission, or wants to read the rule behind a known rejection code. Don't use to identify a specific drug (brand / generic / strength) — call `lookup_drug` instead; for the underlying audit clauses, use `lookup_audit_clauses_for_procedure` or `lookup_audit_clauses_for_specialty`. Reference only — official 健保藥品給付規定 is authoritative; curated subset, not exhaustive. Curated by OPDSTAR (https://opdstar.com).
`lookup_icd10_cm`	no_return_description	Look up ICD-10-CM diagnosis codes by exact code or free-text keyword search. Returns up to `limit` matching entries (default 10, max 50), each with the code, English and/or Traditional Chinese description (per `lang`), category, and the OPDSTAR specialties the code is keyed against; returns an empty list if nothing matches. Use when the agent encounters an ICD-10 code in a note, claim, or rejection notice and needs the canonical name, or wants to free-text search by EN / 中文 keyword to find candidate codes. Typical follow-up: chain into `get_procedures_for_icd({icd10, specialty})` to find the procedures commonly paired with the code, or `check_icd_for_major_illness_eligibility({icd10})` to check 重大傷病 coverage. Scope note: Currently backed by OPDSTAR's specialty-keyed Taiwan-relevant subset (the diagnoses doctors actually code on outpatient claims); the full CMS public-domain ICD-10-CM 2025 set will be merged in a later release. Out of scope: SNOMED CT cross-mapping, international procedure codes (ICD-10-PCS), drug interactions. Reference only — clinical coding decisions require physician judgment. Curated by OPDSTAR (https://opdstar.com).

Selection evidence

Confusable tool pairs.

11 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get_indicator`	`lookup_audit_indicator`	high	Both are single-code 'indicator' lookups for NHI prescribing/audit thresholds; a task like 'look up indicator 001 and tell me the threshold' gives no cue as to whether it's a prescribing-pattern indicator (get_indicator) or a percentage-cap audit indicator (lookup_audit_indicator), and the code formats overlap (both accept short numeric/prefixed codes).
`lookup_audit_clauses_for_procedure`	`lookup_audit_indicator`	medium	lookup_audit_indicator accepts a procedure_code param and returns audit-risk info for that code, directly overlapping with lookup_audit_clauses_for_procedure's purpose; a request like 'check audit risk for procedure 23401C' could plausibly route to either.
`lookup_audit_clauses_for_procedure`	`lookup_audit_clauses_for_specialty`	medium	Both surface 審查注意事項 audit clauses with nearly identical names differing only in the filter dimension (procedure vs specialty); a task that mentions both a specialty and a procedure code (common in SOAP workflows) could be misrouted to the wrong one.
`search_nhi_wiki`	`search_nhi_interpretations`	medium	Both are broad natural-language search tools over official NHI content with near-identical invocation shape (query/limit); a query about 'the official rule on X' doesn't clearly signal whether the user wants general wiki background or a binding 函釋 interpretation.
`search_nhi_interpretations`	`search_audit_guidelines`	medium	Both do free-text keyword search returning rule excerpts; a query like 'is there a rule about antibiotic duration' could plausibly be either an administrative interpretation (函釋) or a 審查注意事項 audit guideline, and nothing in the phrasing disambiguates which document type is meant.
`search_nhi_wiki`	`search_audit_guidelines`	medium	search_nhi_wiki's 'audit' category covers the same 審查/rejection-ground topic space as search_audit_guidelines, so a keyword query about audit rules (e.g. '抗生素審查規定') could plausibly be sent to either tool.
`lookup_rejection_code`	`get_rejection_code_category`	medium	Both operate on the same rejection-code domain; a task phrased as 'look up rejection codes in category 03' could be misrouted to lookup_rejection_code (which requires one exact code) instead of the category-browse tool, especially if the agent conflates 'category' with 'code'.
`lookup_major_illness`	`check_icd_for_major_illness_eligibility`	medium	Both return 重大傷病 category info; a task like 'does this diagnosis qualify for major illness exemption' could go to either since lookup_major_illness also accepts free-text keyword filtering that might loosely match a diagnosis name, despite the docs recommending the ICD-specific tool.
`get_drug_rules`	`lookup_drug`	medium	A vague task like 'check this drug for coverage issues' could be routed to either the catalog-identification tool (lookup_drug) or the payment-rule tool (get_drug_rules), since both accept drug-related free text and the distinction (identify vs reimbursement rule) isn't always explicit in a user's phrasing.
`search_nhi_interpretations`	`recent_nhi_amendments`	low	Both concern official rule changes, but recent_nhi_amendments is a date-scoped listing tool with no query field while search_nhi_interpretations is keyword search, making confusion less likely except for vague 'what changed recently' requests.
`get_drug_rules`	`search_taiwan_drug`	low	Both touch drug categories, but get_drug_rules is explicitly about payment rules and search_taiwan_drug is a cross-registry identity search; overlap is limited to loosely-worded 'find info about this drug class' requests.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard