Public assessment

PantelisGeorgiadis/dicomweb-mcp-server (dicomweb-mcp-server)

dicomweb-mcp-server · v0.0.4 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 95→89, token cost up 3%, unconfirmed writes 100%→100%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

30.0 / 30

30.0 out of 30

02Legibility

26.6 / 30

26.6 out of 30

03Economics

19.3 / 20

19.3 out of 20

04Discoverability

12.3 / 20

12.3 out of 20

Highest-impact fix

Estimated gain +8 points

Make target tools discoverable on the first call

Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.

Description evidence

Defects and rewrites.

6 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`find-instances`	no_return_description	Searches DICOM instances within a single series and returns the matching instances sorted by Instance Number ascending. Does not retrieve image data. Requires Study and Series Instance UIDs from find-studies and find-series.
`find-structured-reports`	no_return_description	Finds and returns all Structured Report (SR) DICOM instances in a study by searching for SR-modality series and filtering by SR SOP Class UIDs. Requires a Study Instance UID from find-studies.
`find-encapsulated-pdf-reports`	no_return_description	Finds and returns all Encapsulated PDF DICOM instances in a study by searching for DOC-modality series and filtering by EP SOP Class UIDs. Requires a Study Instance UID from find-studies.
`get-structured-report-text`	no_return_description	Retrieves a Structured Report (SR) instance and returns its content converted to human-readable text. Does not retrieve image data. Requires Study, Series, and SOP Instance UIDs from find-structured-reports.
`get-encapsulated-pdf-report-text`	no_return_description	Retrieves an Encapsulated PDF instance and returns its content converted to human-readable text. Does not retrieve image data. Requires Study, Series, and SOP Instance UIDs from find-encapsulated-pdf-reports.
`get-instance-metadata`	no_return_description	Retrieves a DICOM instance and returns its metadata converted to human-readable text. Does not retrieve image data. Requires Study, Series, and SOP Instance UIDs from find-instances.

Selection evidence

Confusable tool pairs.

6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get-structured-report-text`	`get-encapsulated-pdf-report-text`	medium	Both tools convert a report instance to human-readable text with identical input schemas; a task like 'get the report text' or 'retrieve the report as readable text' is ambiguous about whether the report is an SR or an encapsulated PDF, so the agent could pick the wrong converter.
`find-structured-reports`	`find-encapsulated-pdf-reports`	medium	Both tools 'find all reports in a study' with nearly identical descriptions and inputs; a natural task such as 'find the reports for this study' leaves the report type unspecified, forcing the agent to guess between SR and PDF.
`find-encapsulated-pdf-reports`	`get-encapsulated-pdf-report-text`	low	A user task like 'get the PDF report' is ambiguous between locating the report instances (find) and extracting their readable text (get), since both tools concern encapsulated PDF reports.
`find-structured-reports`	`get-structured-report-text`	low	'Get the structured report' can be read as locating the SR instances or converting one to text; the shared 'structured report' vocabulary makes the find/get action ambiguous.
`get-encapsulated-pdf-report-text`	`get-instance-metadata`	low	Both retrieve a DICOM instance and convert it to human-readable text; for a generic 'get the text of this instance' task where the agent does not know the SOP class, it could choose the generic metadata tool instead of the PDF-specific one.
`get-structured-report-text`	`get-instance-metadata`	low	Both convert an instance to human-readable text; a task asking to read out a generic instance's content could be satisfied by the generic get-instance-metadata instead of the SR-specific converter.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard