01Safety
0.0 / 30
What changed in the harness
Selection accuracy 91%, destructive-action safety rate 0% (baseline only -- no rewrite pass applied).
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.8 / 30
03Economics
9.0 / 20
04Discoverability
17.0 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
0 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
| No description defects were flagged in this assessment. | ||
Selection evidence
Confusable tool pairs.
5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
research_company |
compose_thesis |
high | Both are paid single-company tools taking the same `name` param and returning near-identical structures (sector, sectorRank, peers[5], period, citation). Tasks like 'build me a research dossier / thesis for Supabase' or 'prepare memo prep on X' are genuinely ambiguous — the only distinguishing phrase is 'dossier with peers' vs 'structured thesis with strengths', which is easy to conflate. |
get_trending_startups |
search_startups_by_sector |
medium | A task like 'what's trending in fintech?' / 'hot AI startups this week' mixes cross-sector-trending language with a named vertical. Agents may call the cross-sector trending tool and ignore the sector intent, or call the sector tool for what was meant as a global top-20 view. The paraphrases 'trending' vs 'in <sector>' collide exactly here. |
get_trending_startups |
get_startup_signal |
medium | Tasks like 'is Roboflow trending?' or 'what's hot on Supabase' are ambiguous: one tool returns the global top-20 list, the other answers whether a named company is trending via signalType. When the user names a company while using trending/velocity language, an agent may pick the list tool (which has no params) or the lookup tool, getting the wrong granularity. |
shortlist_signals |
compare_signals |
medium | 'Compare the top 5 signals in devtools' is ambiguous: compare_signals requires 2-5 pre-named companies while shortlist_signals is the discovery/ranking tool. A user asking to 'compare' or 'shortlist' filtered signals can push an agent to compare_signals with an unnamed set (which then fails/returns notFound), or to shortlist when a head-to-head of named startups was meant. |
compose_thesis |
deep_dive_scan |
medium | Tasks like 'write a deep-dive thesis on the fintech sector' are ambiguous: compose_thesis requires a single company name while deep_dive_scan is the sector-wide segmented scan. Both are paid tools for 'deeper' output and each cross-references the other in its DO NOT USE list, so an agent facing 'sector thesis / deep dive' language may pick the wrong one and hit a required-param mismatch. |
Compare the field