0.0 / 30
What changed in the harness
Selection accuracy 95→100, token cost up 16%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
01Safety
02Legibility
19.3 / 30
03Economics
17.1 / 20
04Discoverability
13.5 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
27 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
get-youtube-transcript |
params_unexplained no_return_description |
Extract the transcript of a YouTube video for use as plain text. Use for captioning, summarization, or subtitle retrieval. Optionally include timestamps in the output, control how many timestamps are grouped together per line, and request a transcript in a preferred language when available. Returns the transcript text, with each line optionally prefixed by its timestamp when includeTimestamps is true. |
search |
name_restates_behavior params_unexplained no_return_description |
Perform a Google web search and return the matching results, optionally the full scraped content of each result page. Use to find web pages, gather current information, or collect full-page content for analysis. Control result localization via country, location, and language, filter by dateRange, and paginate with page. When scrapeResults is true, fetch the contents of each result's page instead of the summary listing, limited to numResultsToScrape pages, rendered as markdown, html, or screenshot per scrapeOptions. Returns a list of search results and any scaped page content. |
get-autocomplete |
params_unexplained no_return_description |
Retrieve Google autocomplete suggestions for a query, useful for discovering search phrasing, related terms, and keyword variants. Localize suggestions using location, country, and language. Returns a list of suggested query strings. |
search-maps |
name_restates_behavior params_unexplained no_return_description |
Search Google Maps for businesses and locations and get their details. Supply coordinates with zoom to search an area, or use placeId or cid to resolve a specific place, and paginate with page. Returns matching places with their business information such as name, address, and ratings. |
search-places |
name_restates_behavior params_unexplained no_return_description |
Search for places and return detailed business information such as contact details, ratings, and address. Localize results with country, location, and language, and paginate with page. Returns matching places with detailed business data. |
search-news |
name_restates_behavior params_unexplained no_return_description |
Search for news articles across multiple sources and return the matching stories. Localize results with country, location, and language, filter by publication date range with dateRange, and paginate with page. Returns a list of news articles with headline and source details. |
get-google-reviews |
no_return_description |
Retrieve Google reviews for a business or place, useful for reputation analysis and customer feedback. Identify the target via keyword, cid, or placeId; set how many reviews to fetch, the sortBy order, and the language, with location providing geolocation context for disambiguation. Returns the requested reviews with rating, author, text, and timestamps. |
scrape |
no_return_description |
Extract and parse content from any web page for downstream analysis. Render JavaScript when renderJs is true, optionally clean the output, and choose the format (markdown, html, or screenshot). Returns the page content in the requested format. |
crawl |
name_restates_behavior params_unexplained no_return_description |
Crawl a website starting from a given URL, following links recursively and extracting content from each page. Limit the crawl with limit (maximum pages) and depth (how many link levels deep), and choose the output format. Returns extracted content for all pages visited. |
extract |
name_restates_behavior params_unexplained no_return_description |
Extract structured data from a web page using AI-powered instructions, converting unstructured page content into fields defined by a schema. Returns the extracted data as structured records matching the provided schema. |
screenshot |
name_restates_behavior params_unexplained no_return_description |
Capture screenshots of web pages with full control over viewport size, image format, and rendering behavior. Use deviceScaleFactor for pixel density (retina-like captures), clipRectangle to crop a specific area, blockCookieBanners to suppress consent dialogs, and wait or renderJs to control page loading. Returns an image file in the requested format. |
doc-to-text |
params_unexplained no_return_description |
Convert various document formats to plain text for processing. Supply input as a url or base64-encoded file via inputMethod, and limit work to specific pages. Returns the document content as plain text. |
convert-to-pdf |
name_restates_behavior params_unexplained no_return_description |
Convert various file formats to PDF, supplied either by url or as base64-encoded content. Returns the converted document as a PDF file. |
merge-pdfs |
params_unexplained no_return_description |
Combine multiple PDFs into a single document, in the order given. Supply PDFs by url or base64, optionally set document metadata (title, author, subject, keywords), PDF/A or PDF/UA compliance, and a requestSource. Returns the merged PDF file. Note: the purpose of requestSource is not evident from the available schema. |
trim-video |
name_restates_behavior params_unexplained no_return_description |
Trim a video to a specific duration by providing the videoUrl and start/end timestamps in HH:MM:SS. Returns the trimmed video clip. |
extract-document |
name_restates_behavior params_unexplained no_return_description |
Extract structured data from one or more documents based on a prompt, supplied as urls or base64. Optionally return the results as JSON via jsonMode. Returns the extracted data according to the prompt. |
extract-image |
name_restates_behavior params_unexplained no_return_description |
Extract structured data from images based on a prompt, supplied as urls or base64. Optionally return the results as JSON via jsonMode. Returns the extracted information from the images. |
extract-audio |
name_restates_behavior params_unexplained no_return_description |
Extract structured data from an audio file based on a prompt, supplied as a url or base64. Optionally return the results as JSON via jsonMode. Returns the extracted information from the audio. |
extract-video |
name_restates_behavior params_unexplained no_return_description |
Extract structured data from a video based on a prompt, supplied as a url or base64. Optionally return the results as JSON via jsonMode. Returns the extracted information from the video. |
read-pdf-metadata |
params_unexplained no_return_description |
Read metadata such as title, author, and subject from PDF files supplied by url or base64. Returns the metadata for each PDF. Note: the purpose of requestSource is not evident from the available schema. |
write-pdf-metadata |
name_restates_behavior params_unexplained no_return_description |
Write metadata such as title, author, subject, keywords, creator, and producer to PDFs supplied by url or base64, producing modified copies. Returns the PDFs with the updated metadata. |
generate-agent-completion |
params_unexplained no_return_description |
Generate an AI text completion from a conversation built from messages, driven by the configured behavior of a specific agentId. Optionally parse the response as JSON and supply a threadId to preserve conversation history across calls. Returns the generated completion text (or parsed JSON). Note: the correct agentId values are not evident from the schema. |
search-knowledge-base |
name_restates_behavior params_unexplained no_return_description |
Search a knowledge base for relevant information matching a query, returning a configurable number of results. Returns matching knowledge base entries. Note: the correct knowledgeBaseId values are not evident from the schema. |
add-to-knowledge-base |
name_restates_behavior params_unexplained no_return_description |
Add new text content as a named resource to a knowledge base so it becomes searchable later. Returns confirmation of the added resource. Note: the correct knowledgeBaseId values are not evident from the schema. |
generate-ai-image |
name_restates_behavior no_return_description |
Generate AI images from text prompts with configurable model and parameters (seeds for reproducibility, output count, aspect ratio, format, quality, inference steps, and guidance scale). Returns the generated image(s). |
run-js-code |
no_return_description |
Executes JavaScript code in an isolated sandbox, optionally running npm install commands first to prepare dependencies. Returns the executed code's output; if parseJson is true, the output is parsed and returned as JSON. The timeout and memory parameters cap execution duration and memory usage. |
run-python-code |
params_unexplained no_return_description |
Executes Python code in an isolated sandbox, optionally running pip install commands first to prepare dependencies. Returns the script's output; if parseJson is true, the output is parsed and returned as JSON. When saveOutputFiles is true, files produced by the script are persisted and returned so they can be accessed afterward. The timeout and memory parameters cap execution duration and memory usage. |
Selection evidence
Confusable tool pairs.
8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
search-maps |
search-places |
high | Both tools search for locations/businesses with nearly identical schemas (query, language, page). A task like 'find restaurants with details in Paris' maps equally well to both, since search-maps is 'locations and businesses' and search-places is 'places with detailed business information'. |
search |
search-news |
medium | A task like 'search Google for recent news about AI' fits the generic web search tool as well as the news-specific one; the generic 'search' has no media/domain qualifier, so the agent may pick it over search-news. |
search |
search-maps |
medium | 'Search for nearby coffee shops' or 'find X on Google' could be routed to the generic web search tool instead of the maps tool because the generic tool is named just 'search' and both take a query. |
search |
search-places |
medium | A task like 'search for businesses/places near me' lacks the word 'map', so the agent may choose generic 'search' rather than search-places, which is the domain-specific match. |
extract |
extract-document |
medium | 'Extract structured data from this document/PDF URL with this schema' fits both: 'extract' takes a URL plus schema and is 'AI-powered', while extract-document takes files plus a prompt. The overlapping structured-data intent makes the generic 'extract' a plausible wrong pick. |
extract |
extract-image |
medium | A task like 'extract structured data from this image URL' could cause the agent to use generic 'extract' (URL + schema) instead of extract-image, since the user supplies a URL and schema-style instructions rather than an explicit image prompt. |
extract-document |
extract-image |
medium | 'Extract data from these files' or a scanned PDF (which is an image file) fits both descriptions; the agent must infer media type from the task, and a generic 'extract data from these attached files' task is ambiguous between documents and images. |
run-js-code |
run-python-code |
medium | A task like 'execute this script/sandbox code' where the user does not name the language is ambiguous; both tools are structurally identical sandboxes that differ only by the language in their descriptions, so an agent may pick the wrong one. |
Compare the field