01Safety
30.0 / 30
What changed in the harness
Selection accuracy 86%, destructive-action safety rate 100% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.2 / 30
03Economics
18.9 / 20
04Discoverability
17.0 / 20
Highest-impact fix
Estimated gain +3 pointsMake target tools discoverable on the first call
Clarify tool names, decision boundaries, and required argument schemas so an agent can choose and construct the target call without exploratory steps.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
xcomet_evaluate |
xcomet_batch_evaluate |
medium | A request like 'evaluate the quality of these translations' without specifying single vs. multiple pairs could plausibly target either tool; if the task involves just one source/translation pair but is phrased generically (e.g. 'check translation quality for this document'), an agent might wrongly reach for batch_evaluate, or vice versa if 'these' implies plural but only one pair is actually provided. |
xcomet_evaluate |
xcomet_detect_errors |
low | Both operate on a single source/translation pair and xcomet_evaluate's output includes an 'errors' field similar to xcomet_detect_errors, so a task like 'find issues in this translation' could ambiguously map to either tool, though detect_errors' explicit focus on error detection and evaluate's explicit focus on scoring make the distinction fairly clear. |
Compare the field