01Safety
0.0 / 30
Public leaderboard
Public assessment
AutomateLab-tech/citation-intelligence (@automatelab/citation-intelligence)
What changed in the harness
Selection accuracy 96%, destructive-action safety rate 0% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.7 / 30
03Economics
8.5 / 20
04Discoverability
12.0 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
12 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
audit_sitemap |
audit_sitemap_map |
high | Names differ by one token ('map') but semantics diverge sharply: audit_sitemap runs a live heuristic citability score on every sitemap URL (no keys needed), while audit_sitemap_map only cross-references the sitemap against an already-primed citation cache. A task like 'audit my sitemap against citations' plausibly triggers the wrong one, especially since the cache-dependent tool silently returns empty results if unprimed. |
domain_am_i_cited |
domain_cited_for |
high | Both answer 'is my domain cited' for a set of queries, but am_i_cited fires live API calls against supplied queries while cited_for reads cached history and returns whatever queries are already on file. A phrasing like 'what queries is my domain cited for' could be routed to either, with very different cost/freshness tradeoffs. |
domain_cited_for |
domain_cited_for_diff |
medium | domain_cited_for_diff is a variant of domain_cited_for that adds time-window comparison; near-identical names and both cache-only. A request like 'show citation changes for my domain' could plausibly select the plain listing tool instead of the diff tool, or vice versa if the user just wants a current snapshot. |
domain_am_i_cited |
domain_cited_for_diff |
medium | Both are framed around tracking whether/how a domain's citation status changes, one via live re-check and one via cache diff over time. A task like 'did my citation status improve' is ambiguous between running a fresh live check and diffing historical cache windows. |
signals_gsc_gap |
signals_bing_gap |
medium | Structurally identical tools (join search-console-style rank data with am_i_cited) differing only by data source (Google vs Bing) and required credentials. A generic task like 'find queries where I rank well but aren't cited by AI' doesn't specify which search engine's console data to use, making the choice ambiguous unless the agent infers from available credentials. |
competitors_compare |
competitors_compete |
high | Near-identical names ('compare' vs 'compete') with overlapping purpose (competitive citation-signal comparison) but different inputs: compare takes a raw URL list and does heuristic scoring only, while compete takes a query+your_url and does a live citation check plus comparison. A task like 'compare my page to competitors for this query' plausibly maps to either tool. |
competitors_compete |
competitors_canonical_set |
medium | Both fan a single query across AI engines to identify competitors, but compete requires your_url and benchmarks against it, while canonical_set has no your_url and just ranks competitor domains. A task like 'who are my AI-search competitors for this query' could be routed to either depending on whether the agent registers the need for a benchmark URL. |
citations_check |
citations_predict |
medium | Both are framed around 'will this be cited', but citations_check queries a live AI engine with a search query while citations_predict is a pure heuristic scorer that takes a URL and never calls an LLM. A task like 'check if this page gets cited' is ambiguous about whether the user wants a live query test or a heuristic score. |
citations_check |
citations_provenance |
medium | citations_provenance is essentially citations_check fanned out across multiple engines with consensus reporting. A task like 'check which sources are cited for this query across AI engines' could be satisfied by either the single-engine check or the explicit multi-engine provenance tool. |
citations_check |
citations_evidence |
medium | citations_evidence wraps citations_check and adds snippet extraction from the raw answer. A task like 'check what's cited and why' is ambiguous between the base citation list and the evidence-extraction variant, especially since both share the same query input and most parameters. |
panel_track |
panel_run |
medium | panel_track manages (save/load/list) a saved query set while panel_run actually executes it against an AI engine and records a snapshot. A task like 'run my tracked query panel' could be misrouted to panel_track (e.g. action=load) instead of panel_run, since both operate on the same named panel concept. |
signals_ai_overview |
signals_answer_box |
low | Both concern where/whether a URL shows up in an AI-generated answer, but signals_ai_overview checks Google's specific AI Overview SERP feature while signals_answer_box locates citation position within a general AI engine's raw answer text. A vague task like 'does my content show up in the AI answer' could plausibly go to either. |
Compare the field