01Safety
30.0 / 30
What changed in the harness
Selection accuracy 98→98, token cost up 2%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.7 / 30
03Economics
17.1 / 20
04Discoverability
18.8 / 20
Highest-impact fix
Estimated gain +3 pointsReduce the tool-definition and task token payload
Consolidate overlapping tools and trim repetitive schema text while preserving behavior, parameter, and return semantics.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
auditBibliography |
params_unexplained |
Verify a WHOLE bibliography in one call — the batch counterpart to verifyCitation. Each entry runs the same fabrication check (real, resolvable identifier paired with a title that does NOT match the resolved paper; Topaz et al., Lancet 2026) plus a retraction lookup, and the tool returns a per-entry verdict table and a corpus summary. Use when the user pastes a reference list, a .bib / .ris file, or asks to 'check all these citations at once' / 'audit my bibliography' / 'which of these references are fake or retracted'. Input: EITHER `bibliography` (raw BibTeX / RIS / CSL-JSON text — format auto-detected) OR `claims` (an array of pre-parsed citation objects), not both. Each `claims` item requires `title` (the cited title) and carries whatever identifiers the cited reference has via the optional fields `doi`, `pmid`, `pmcid`, `isbn`, `arxiv`, `issn`, `ads`, or `whoIrisUrl` (the verifier uses the first identifier in priority order doi > pmid > pmcid > arxiv > ads > isbn > issn > whoIrisUrl, as in verifyCitation); optional refiner fields are `author` (first-author family name as cited), `year` (cited publication year — a large gap from the resolved record lowers confidence), and `container` (journal or container name as cited, a soft signal). Capped at 25 entries per call; excess is dropped and reported via `truncated`. `checks` defaults to ['retraction'] (pass [] to skip); `screenWithLlm` opt-in per entry (same auth gating as verifyCitation). Returns: { format, entries: [{ index, sourceKey?, status: 'ok'|'error', verdict: 'matched' | 'mismatch' | 'not_found' | 'ambiguous', confidence, matched, mismatches, retraction: { checked, doi, isRetracted, hasCorrections, hasConcern, notices } | null, _provenance }], parseErrors: [{ index, error, message }], truncated, summary: { total, matched, mismatch, ambiguous, not_found, errored, retracted } }. Per-entry leniency: one entry that fails to resolve becomes status:'error' without failing the batch. This audits citation IDENTITY (does each identifier resolve to the claimed work, and is it retracted) — it does NOT check whether a source supports the claim it is cited for. Read-only and idempotent. Works anonymously for the non-LLM path; SCHOLAR_API_KEY (a free ssk_ key from https://scholar-sidekick.com/account) or a paid RapidAPI tier raises rate limits and enables the optional LLM screen. |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
formatCitation |
exportCitation |
medium | Both accept the same identifiers and can produce styled reference text (exportCitation with format='txt'+style renders an APA-style bibliography), so a task like 'format this reference list / make a bibliography for these papers' is genuinely ambiguous about whether the user wants in-line citation strings or a downloadable file, and the docs' boundary ('paste-ready in-line text' vs 'file for a reference manager') is easy for an agent to miss in casual phrasing. |
checkRetraction |
checkOpenAccess |
medium | Both are near-identical single-identifier tools (resolve to DOI, return doi/reason/result, no_doi failure), so an underspecified task like 'check this paper','is this work okay to use?', or 'what's the status of this paper?' does not reveal whether the user means retraction/standing (checkRetraction) or free-to-read availability (checkOpenAccess), and neither description dominates such phrasing. |
verifyCitation |
formatCitation |
low | Both accept citation inputs, and a task like 'fix this citation' or 'produce a correct citation for this paper' could be read either as reformatting identifiers (formatCitation) or as validating the claimed title against the resolved identifier (verifyCitation); the intended verb (format vs verify) is usually clear, so confusion is possible but unlikely. |
Compare the field