01Safety
0.0 / 30
What changed in the harness
Selection accuracy 89→84, token cost up 3%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.8 / 30
03Economics
18.7 / 20
04Discoverability
14.9 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
scrape_website |
no_return_description |
Extract and return the content of a single URL. Returns the scraped content in the chosen "output_format": "markdown" (default), "html", or "text" need no extra config; "json" returns structured data and requires either a "parser" or an "llm_extract" schema (do not request "json" on its own). Supports JavaScript rendering. Optional "country" code (e.g., US, GB, CA) loads the request from that location; "wait_before_scraping" (0-10000 ms) waits before scraping to handle dynamic content; "parser" selects a specialized extraction template (e.g., "@olostep/amazon-product"); "url_to_scrape" is an alias for "url". |
Selection evidence
Confusable tool pairs.
3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
create_map |
get_website_urls |
high | Both are pure URL-discovery tools with near-identical descriptions ('URL discovery only - does NOT scrape', map/list links on a website). A task like 'find all URLs on example.com' or 'map the site's blog links' could select either; get_website_urls additionally requires a search_query users may not mention, making create_map an attractive wrong pick. |
get_crawl_results |
get_batch_results |
medium | Both retrieve status/content of an async job from an ID. For a task like 'check the status of my scrape job' or 'get the results of my crawl', an agent must recall whether the job was a crawl or batch; if that isn't tracked, the generic 'get job results' description makes either tool a plausible choice. |
create_map |
create_crawl |
medium | 'Map this website' naturally maps to create_map, but users often mean 'crawl the whole site and get all content'; create_crawl's description positions it as preferred for whole-site scraping while create_map explicitly does NOT scrape. The wording can drive selection of the wrong member depending on whether the deliverable is a URL list or scraped content. |
Compare the field