Public assessment

MyMedi-AI/mymedi-ai-mcp-server (@mymedi-ai/mcp-server)

mymedi-ai-mcp-server · v1.5.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 91→87, token cost up 4%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

21.3 / 30

21.3 out of 30

03Economics

16.7 / 20

16.7 out of 20

04Discoverability

13.2 / 20

13.2 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

13 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`code_lookup_basic`	no_return_description	Look up basic metadata for a medical code. Returns the code with its codeType, description, category, and isActive flag. Basic metadata only — the paid code_lookup adds full metadata. Free, no API key required.
`order_readiness_checklist`	no_return_description	Generate a blank pre-delivery requirements checklist for a HCPCS DMEPOS code. Returns the universal standard written order (SWO) elements (42 CFR 410.38(d)), whether the code requires a face-to-face encounter and written order prior to delivery (F2F/WOPD), and whether it is on the Medicare Required Prior Authorization List. Requirement definitions only — PHI-free, never send patient data. Free, no API key required.
`modifier_advisor`	no_return_description	Provide editorial guidance on DMEPOS billing modifiers, including the KX/GA/GY/GZ medical-necessity and liability family, RR/NU/UE rental-vs-purchase, capped-rental month markers (KH/KI/KJ), and RT/LT laterality. Returns the modifiers relevant to the given HCPCS code, DMEPOS category, or scenario phrase, with guidance on when each applies. Pass a HCPCS code to scope guidance to that item's DMEPOS category, or pass a category directly; add a scenario phrase (e.g., ABN on file, bilateral) to surface the relevant modifiers. Original editorial content, not payer policy. PHI-free. Free, no API key required.
`code_suggest`	no_return_description	Suggest ICD-10/CPT/HCPCS codes from a clinical description. Returns a ranked list of matching codes with descriptions, ranked by matched-term coverage and relevance. Term-based search over the 81K-code CMS database. Works with natural sentences (e.g., patient with obstructive sleep apnea prescribed CPAP). Automatically refunds the call when nothing matches.
`code_validate`	params_unexplained	Validate a medical code for correctness, active status, and context. Returns warnings and errors. codeType scopes validation to a code system (icd10, cpt, or hcpcs) when provided; context optionally supplies a dateOfService (YYYY-MM-DD) used for temporal validation.
`code_validate_batch`	params_unexplained	Validate a list of medical codes for correctness, active status, and optional date-of-service context in one call. Returns per-item results mirroring code_validate (valid, active, warnings, errors, codeDetails). codeType applies a code system to every code; context supplies a dateOfService (YYYY-MM-DD) used for temporal validation of every code. Priced per code — $0.005 × number of codes, max 25 per call. An invalid code is a billable answer (valid:false), same as the single validate.
`ner_extract`	no_return_description	Extract medical named entities from clinical text. Returns the identified entities — ICD-10 codes, CPT codes, dates, medications, and 12 entity types — each with a confidence score. entityTypes optionally filters results to the requested entity types only.
`drug_lookup`	no_return_description	Look up drug information by name or field. Returns the drug's label data, adverse events, and related diagnosis codes, matched via searchField (brand_name, generic_name, product_ndc, or substance_name). Source: OpenFDA (public domain).
`fee_schedule_lookup`	no_return_description	Look up the CMS DMEPOS fee schedule for a DME/orthotic/prosthetic HCPCS code. Returns rental (RR) vs purchase (NU new / UE used) rates, non-rural vs rural — exact per-state rates when a state is given, or national min–max ranges when omitted; modifier narrows results to rental or purchase. Source: CMS DMEPOS Fee Schedule DME26-B (Apr 2026). For professional-service (CPT) payment use code_reimbursement instead.
`drug_rxnorm`	no_return_description	Look up a drug in NIH RxNorm. Returns the normalized terminology (RxCUI) for the drug name, and — when checkInteractions lists other drugs — clinical drug-drug interaction results with severity ratings. Source: NIH RxNorm (public domain).
`disease_surveillance`	no_return_description	Look up disease surveillance data from the CDC National Notifiable Diseases Surveillance System (public domain). Query by a disease condition name or an ICD-10 code, optionally filtering by a 2-letter state code. Returns case counts and trends for the specified condition and geography.
`provider_search`	params_unexplained no_return_description	Search the NPI provider directory for healthcare providers. Supply firstName and lastName to find an individual, or organizationName to find an organization; optionally filter by taxonomy (specialty code), city, and 2-letter state code, and pass limit to cap the number of results (default 10, max 50). Returns matching providers with their NPI records including name, location, and taxonomy.
`account_status`	no_return_description	Check your MyMedi-AI account without consuming credits. Returns your current credit balance, its USD equivalent, transaction count, recent transactions, and last activity, using your connector sign-in (or X-API-Key when self-hosted).

Selection evidence

Confusable tool pairs.

19 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`code_lookup`	`code_lookup_batch`	medium	A task like 'look up these medical codes' (or fee/status info for several codes) fits both the single-code tool (called repeatedly) and the batch tool; nothing in the schema forces the choice unless the user states a count.
`code_lookup_basic`	`code_lookup`	medium	Both answer 'what is this code' with description, category, and active status; a request like 'look up M79.3' matches the free basic tool and the paid full tool equally, differing only in DMEPOS fee extras.
`code_validate`	`code_validate_batch`	medium	Validating a set of codes could be done one-by-one or in the batch variant; a task like 'validate these codes / check all are active' leaves the single-vs-batch choice ambiguous.
`code_lookup_basic`	`code_lookup_batch`	low	'Look up basic info for these codes' could be satisfied by iterating the free basic lookup or by the list-based batch tool; neither is clearly canonical.
`code_lookup`	`code_crossref`	medium	code_lookup returns 'related codes' while code_crossref returns cross-system related codes, so a task asking for codes related to an ICD/CPT/HCPCS code fits both descriptions.
`code_lookup_batch`	`fee_schedule_lookup`	low	code_lookup_batch embeds the DMEPOS fee schedule per HCPCS code, overlapping fee_schedule_lookup, so 'get DMEPOS fees for these codes' could pull either tool.
`reimbursement_basic`	`code_reimbursement`	medium	Both return Medicare payment amounts for a code via the CMS PFS; 'how much does Medicare pay for 99213' matches both, differing mainly in RVU detail versus DMEPOS ranges.
`code_validate`	`claims_validate`	low	Both are 'validate' tools; wording like 'make sure my claim is correct' could pull the claim-level validator while 'are these codes valid' pulls the code validator, and the phrases overlap.
`drug_interactions`	`drug_rxnorm`	medium	Both accept two or more drugs and check drug-drug interactions (FAERS co-report signals vs RxNorm clinical with severity); 'check interactions between warfarin and aspirin' fits both.
`drug_lookup`	`drug_enrich`	medium	Both take a drug name and return OpenFDA drug information; 'get info on atorvastatin' matches both, differing only in extra fields (adverse events/Dx codes vs AI analysis).
`drug_lookup`	`drug_rxnorm`	medium	Both are drug-name lookups (label/adverse events vs normalized RXCUI terminology); a bare 'look up this drug' request doesn't disambiguate them.
`drug_rxnorm`	`drug_enrich`	low	Both are single-drug information lookups sharing the same drugName input; 'tell me about rivaroxaban' matches both descriptions closely.
`drug_interactions`	`drug_enrich`	low	A singular request like 'what interactions does metformin have' could misleadingly pull the multi-drug interactions tool even though enrichment returns interactions for one drug.
`pa_required_check`	`pa_predict`	medium	Both report the CMS Required Prior Authorization List status for a given code, so 'is prior authorization required for E0651' matches both; pa_predict adds an approval rate only when a cohort exists.
`pa_predict`	`pa_status`	medium	Both are prior-authorization checks; 'check the prior authorization' is ambiguous between approval outlook (procedure code) and request status (auth ID).
`pa_required_check`	`pa_status`	low	Both read as 'prior authorization check'; the request could mean a code-level PA-requirement check or the status of an existing PA request.
`pa_predict`	`pa_exposure_report`	low	Both take HCPCS procedure codes and surface CMS PA-list facts; a task about PA outlook over a list of codes could select either the single-code predictor or the catalog report.
`pa_required_check`	`pa_exposure_report`	low	Both answer 'is this code on the CMS Required PA list' — the free single-code check versus the paid multi-code catalog report.
`provider_payments`	`provider_enrich`	low	Both take an NPI and return provider intelligence; a generic 'get me info on this physician' request is ambiguous between the payments dataset and the enriched profile.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard