01Safety
30.0 / 30
What changed in the harness
Selection accuracy 98%, destructive-action safety rate 100% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.4 / 30
03Economics
8.1 / 20
04Discoverability
13.5 / 20
Highest-impact fix
Estimated gain +12 pointsReduce the tool-definition and task token payload
Consolidate overlapping tools and trim repetitive schema text while preserving behavior, parameter, and return semantics.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
ris_dokument |
ris_dokument_abschnitt |
high | Near-identical params (dokumentnummer/url) and shared 'document' vocabulary. A task like 'get the text of section 5 of NOR40052761' can route to ris_dokument_abschnitt for full text, or to ris_dokument when only a section is wanted; only the fine print of the description ('not for direct use... use ris_dokument') disambiguates. |
ris_bundesrecht |
ris_landesrecht |
high | Both are generic Austrian law searches with nearly identical schemas, differing only by federal vs provincial scope. A task like 'search Austrian law for Schadenersatz' or 'find the law on building regulations' omits the federal/state level, making the wrong choice genuinely likely unless the agent knows the topic's level. |
ris_bundesgesetzblatt |
ris_landesgesetzblatt |
medium | Both are 'law gazette' searches; a user asking 'look up the gazette publication of the new law' or 'which gazette printed this' without naming federal vs state level can plausibly land on either tool. |
ris_bundesrecht |
ris_bundesgesetzblatt |
medium | Both target 'federal Austrian legal text.' A task like 'find the federal law as published in the BGBI' or simply 'find the federal regulation text' could route to the consolidated-law searcher when the gazette was meant, or vice versa. |
ris_landesgesetzblatt |
ris_verordnungen |
medium | Both are state-level official publications (state law gazette vs state ordinance gazette). A task like 'find the state gazette where the ordinance was published' or 'search state ordinances' is vague enough that either could be chosen. |
ris_bezirke |
ris_gemeinden |
medium | Both cover locality-level Austrian instruments with overlapping schema (suchworte/titel/bundesland). A task like 'find the local announcement/ordinance about Bauordnung for Graz' could pick municipal law when district announcements were meant, since neither name implies the other's level. |
ris_landesrecht |
ris_verordnungen |
low | Both concern state-level normative text. A vague task like 'find the state regulation on X in Tirol' could pick consolidated state law over the state ordinance gazette; descriptions reduce but do not eliminate the ambiguity. |
ris_bezirke |
ris_verordnungen |
low | Both publish local/state ordinances. A task like 'search for a Verordnung of the district administration in Tirol' could route to the state ordinance gazette, though the respective descriptions ('district authorities' vs 'ordinance gazettes, only Tirol') mostly disambiguate. |
Compare the field