Public assessment

hedging8563/tokenlab-mcp-server (@tokenlabai/mcp-server)

tokenlabai-mcp-server · v0.6.17 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 98→98, token cost up 8%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

19.8 / 30

19.8 out of 30

03Economics

10.2 / 20

10.2 out of 20

04Discoverability

11.9 / 20

11.9 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

27 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`cancel_task`	no_return_description	Cancels a queued asynchronous task when cancellation is supported, returning the outcome of the cancellation attempt so clients know whether to stop polling. Cancellation currently supports queued Seedance video tasks (`seedance-1.5-pro`, `seedance-2.0`, and `seedance-2.0-fast`) while they are still waiting to run. Tasks that are already processing, completed, failed, expired, or unsupported are not cancelled.
`create_3d_model`	params_unexplained	Creates a 3D model generation task using Tripo3D and returns a task ID for polling. `prompt` describes the 3D model; `image` supplies base64 image input or `image_url` supplies a public image URL for image-to-3D; `model` selects the generator (default `tripo-h3.1`); `format` selects the output format (`glb`, `fbx`, `obj`, or `usdz`); `quality` selects `draft`, `standard`, or `high`; `style` hints at compatible 3D model families; `seed` is for deterministic-compatible providers; and `user` identifies the end user.
`create_anthropic_message`	params_unexplained	Creates a model response using Anthropic's native request format and returns the assistant message with usage and stop reason. Supports Claude models with vision and streaming. `temperature` (0-1) controls sampling randomness; `top_p` (0-1) applies nucleus sampling; `top_k` restricts sampling to the top-k most likely tokens; `stop_sequences` halts generation at any listed string. Authentication: Use `x-api-key` header or `Authorization: Bearer` header.
`create_chat_completion`	no_return_description	Creates an OpenAI-compatible chat completion and returns the assistant's message, finish reason, and usage in the OpenAI Chat Completions response shape. For inline image_url data URLs, declare the byte-accurate image MIME type. The MCP boundary corrects recognized PNG, JPEG, WebP, and GIF payloads declared as application/octet-stream and rejects unrecognized generic binary image payloads before sending a billable request.
`create_embedding`	name_restates_behavior no_return_description	Computes an embedding vector, or one vector per input item, representing the input text, and returns the vectors (as `float` or `base64` per `encoding_format`) together with usage. `model` selects the embedding model; `input` accepts text(s) or token arrays; `encoding_format` selects float or base64 output; `dimensions` optionally sets the output vector dimensions; and `user` identifies the end user.
`create_gemini_content`	params_unexplained no_return_description	Generates content using the native Gemini GenerateContent shape and returns the model's response candidates, including generated text, function calls, and other content parts. This route is exposed only when the model details advertise Gemini requests and a same-protocol route is currently available. `systemInstruction` sets the system prompt; `generationConfig` controls sampling and output such as temperature, maxOutputTokens, and responseMimeType; `safetySettings` applies category thresholds. ProtoJSON lowerCamelCase and original proto snake_case field names are preserved. Unknown fields are forwarded best-effort and support is determined by the selected provider. Authentication: Use `?key=` query parameter, `x-goog-api-key` header, or `Authorization: Bearer` header.
`create_image`	name_restates_behavior params_unexplained no_return_description	Generates novel or edited images from a text prompt and returns the generated image(s) as URLs or base64 JSON according to `response_format` (`url` default or `b64_json`). `prompt` describes the image; `model` must be sent explicitly (query GET /v1/models?recommended_for=image for recommendations); `operation` selects text-to-image, image-to-image, or image-edit; `image_url`, `image_urls`, and `reference_image_urls` supply reference images; `n`, `size`, `quality`, `style`, `expand_prompt`, `negative_prompt`, `seed`, `background`, `output_format`, and `output_compression` configure generation for compatible models; `async` returns a task before the final image is ready; and `user` identifies the end user.
`create_image_file`	params_unexplained no_return_description	Generates an image from a prompt using local file uploads and returns the generated image(s) as URLs or base64 data per `response_format`. `model` must be sent explicitly; `prompt` describes the image; `operation` selects text-to-image, image-to-image, or image-edit; `image` uploads a local reference image (pass a local file path); `image_urls` and `reference_image_urls` provide comma-separated reference URLs; `mask_url` supplies a mask; `n` sets the number of images; `quality`, `size`, `aspect_ratio`, `resolution`, `style`, `expand_prompt`, `negative_prompt`, `seed`, `background`, `output_format`, `output_compression`, `compression`, `moderation`, and `async` configure generation for compatible models; and `user` identifies the end user.
`create_multimodal_embedding`	params_unexplained no_return_description	Creates embeddings for multimodal input items and returns the resulting vectors. `input` is a single item or array of items, each specifying `text` to embed and/or image data via `image_base64` with `image_mime_type`, or a public `image_url`; text input is generally available, while image input may require feature enablement. `model` selects the multimodal embedding model, and `dimensions` optionally sets output dimensionality when supported.
`create_music`	params_unexplained	Creates a music generation task using Suno and returns a task ID for polling. `model` selects the Suno model (default `suno_music`); `prompt` describes the music (required for music, lyrics, upload-cover, and upload-extend; omit for add-instrumental); `title` names the track; `tags` describes the track; `action` selects MUSIC or LYRICS-only generation; `negative_tags` lists styles to avoid; `mv` selects the official Suno model version; and `make_instrumental`, `audio_url`, `audio_operation`, `continue_clip_id`, and `continue_at` configure instrumental, cover, extension, and continuation flows.
`create_response`	params_unexplained	Creates a response using the native OpenAI Responses API shape. The model details must advertise the Responses request format and a same-protocol route must be currently available; model names and providers do not imply availability. `temperature` controls sampling randomness; lower values make output more deterministic. Unknown request fields are forwarded on a best-effort basis and remain subject to the selected provider's support.
`create_speech`	name_restates_behavior no_return_description	Synthesizes spoken audio from the input text (Text-to-Speech) and returns the generated audio in the requested `response_format` (common values mp3, opus, aac, flac, wav, and pcm). `model` selects the TTS model (default `tts-1`); `voice` or `voice_id` selects the speaker; `instructions` and `prompt` tune delivery style; `language_code` sets the language; `stream_format` controls direct audio vs SSE delivery; `speed` adjusts the speaking rate; and `temperature` sets sampling for Gemini-compatible routes.
`delete_file`	no_return_description	Deletes a batch file or a bound image/reference file and returns confirmation that the deletion succeeded. When the request includes anthropic-beta: files-api-2025-04-14, the route uses Anthropic Files API mode for the bound workspace.
`edit_image`	params_unexplained no_return_description	Edits an existing image using multipart image uploads, JSON image URLs, or the official JSON images array for supported GPT Image models, and returns the edited image(s) as URLs or base64 data per `response_format`. `prompt` describes the edit; `model` must be sent explicitly; `image_url`, `image_urls`, and `images` provide source images (up to 16 for GPT Image, 3 for xAI Grok Imagine); `mask` supplies an edit mask; and `n`, `quality`, `size`, `aspect_ratio`, `resolution`, `background`, `output_format`, `output_compression`, `compression`, `moderation`, `async`, and `user` configure the edit for compatible models.
`edit_image_file`	params_unexplained no_return_description	Edits an image from local file uploads and returns the edited image(s) as URLs or base64 data per `response_format`. `image` supplies the source image file(s) (up to 16 for GPT Image, 3 for xAI Grok Imagine; pass local file paths, or use the `image[]` alternative multipart field name); `mask` supplies a mask file path; `prompt` describes the edit; `model` must be sent explicitly (e.g., gpt-image-2); and `n`, `quality`, `size`, `aspect_ratio`, `resolution`, `background`, `output_format`, `output_compression`, `compression`, `moderation`, `async`, and `user` configure the edit for compatible models.
`get_model`	params_unexplained no_return_description	Returns the requested model's current details, including supported operations, request parameters, request endpoints, selector limits, and pricing when available. `model` is the ID of the model whose details to retrieve. Non-chat integrations should read these current model-specific request details before creating a request.
`get_model_pricing`	params_unexplained no_return_description	Returns pricing-only detail for a single model. `model` is the ID of the model whose pricing to fetch. Use this endpoint for price explanation, not for non-chat request construction.
`list_files`	name_restates_behavior params_unexplained no_return_description	Returns a paginated list of files (batch files and bound image/reference files) from the Files API. `limit` caps the number of returned files (default 20, max 1000); `after` returns files after the specified cursor and requires the Anthropic Files beta header; `after_id` and `before_id` are Anthropic Files cursors requiring that header; `scope_id` is a reserved Anthropic Files cursor that currently returns an unsupported error; `purpose` filters by file purpose (batch, batch_output, user_data, vision); and `anthropic-beta` with files-api-2025-04-14 selects Anthropic Files API mode.
`list_models`	no_return_description	Returns the currently available models, optionally filtered by `provider`, `tag`, or `category` and sorted with `recommended_for`, formatted according to `view` (`compact` for model selection, `full` for the OpenAI-compatible discovery shape). Use `view=compact` for model selection and `view=full` for the existing OpenAI-compatible discovery shape. Native API calls default to `full`; integrations may choose `compact` as their default. Non-chat recommendations are available through `recommended_for`.
`rerank_documents`	params_unexplained no_return_description	Reranks documents by relevance to a query using semantic similarity and returns the documents ordered by descending relevance with their relevance scores. `model` selects the reranker; `query` is the query to rank against; `documents` are the texts to rerank; `top_n` limits the number of results returned; and `return_documents` controls whether the original document text is included in the results.
`retrieve_file`	name_restates_behavior params_unexplained no_return_description	Returns metadata and details for a stored file from the Files API. `file_id` is the ID of the file to retrieve, as returned by the Files API. Include `anthropic-beta: files-api-2025-04-14` to use Anthropic Files API mode.
`retrieve_file_content`	name_restates_behavior params_unexplained no_return_description	Returns the raw content (bytes) of a stored file from the Files API. `file_id` is the ID of the file whose contents to read.
`transcribe_audio`	name_restates_behavior params_unexplained no_return_description	Transcribe an audio file into text using a Whisper speech-to-text model. The file parameter takes a local file path; model selects the Whisper variant; language sets the ISO-639-1 source language; prompt provides optional guiding context; response_format selects between plain text, JSON, SRT, verbose JSON, or VTT output; temperature (0-1) controls sampling randomness; timestamp_granularities requests word- or segment-level timestamps, which requires verbose_json output. Returns the transcription in the requested format.
`translate_audio`	name_restates_behavior params_unexplained	Transcribe an audio file and translate the recognized speech into English. The file parameter takes a local file path and is processed with the selected Whisper model; prompt optionally guides the transcription; response_format controls the output structure (e.g. JSON, text, SRT); temperature (0-1) tunes sampling randomness. Returns the English translation of the audio in the requested format.
`translate_text`	name_restates_behavior params_unexplained no_return_description	Translate a text string into a target language using the current translation request format. model selects the translation model; text holds the source content; source_language and target_language identify the input and output languages; mime_type (text/plain or text/html) tells the server how to parse the input; user optionally identifies the requester for usage tracking. Returns the translated text in the target language.
`compare_models`	name_restates_behavior params_unexplained no_return_description	Fetch and compare public TokenLab model details and pricing for 2-8 model IDs. Provide the model IDs in models and set include_raw to true to also return unprocessed data. Returns a comparison of capabilities and pricing for the requested models.
`get_api_overview`	no_return_description	Fetch TokenLab's agent-readable API overview describing the service's endpoints and capabilities. Returns the overview content formatted for machine consumption.

Selection evidence

Confusable tool pairs.

8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`create_image`	`create_image_file`	high	Both tools have identical descriptions ('Create image Creates an image given a prompt.') and overlapping schema (model, prompt, operation, size, quality, n). A task like 'create an image of a cat' matches both definitions equally; only a user-supplied local file path versus image URL disambiguates them, so an agent using descriptions alone can pick the wrong member.
`edit_image`	`edit_image_file`	high	Both descriptions are character-identical ('Edit image Edits an image using multipart image uploads, JSON image URLs, or the official JSON images array...'). A task such as 'edit my photo to add text' matches both exactly, and only the input modality (local file path vs URL/JSON) in the schema separates them, creating genuine selection ambiguity.
`create_embedding`	`create_multimodal_embedding`	medium	Both are 'create embeddings' tools accepting text input, and create_multimodal_embedding explicitly notes 'Text input is generally available'. A task like 'create an embedding for this sentence' fits both definitions; an agent must infer whether the user intends the multimodal variant, so a text-only phrasing can lead to the flag/file variant or vice versa.
`get_model`	`get_model_pricing`	medium	get_model's description advertises 'pricing when available', while get_model_pricing is 'pricing-only detail'. A task like 'how much does model X cost' or 'get pricing for model X' matches both tool definitions; an agent may pick the broader get_model instead of the pricing-specific endpoint, or the reverse, when only the model ID is given.
`get_model_pricing`	`get_pricing`	medium	Both are pricing tools (one single-model, one list with provider/tag filters). A natural task like 'get pricing' or 'show me prices for the image models' could select either: get_pricing for the surface list or get_model_pricing for one model's price explanation, so the intended granularity is ambiguous from the wording.
`retrieve_file`	`retrieve_file_content`	medium	retrieve_file returns metadata and retrieve_file_content returns content, but a user task 'retrieve this file' or 'get file abc123' is satisfied by the XML-style wording of both ('Retrieve file' vs 'Retrieve file content'); one takes a bare file_id and the other also requires only file_id, so an agent can fetch the wrong thing.
`create_anthropic_message`	`create_chat_completion`	medium	Both are chat-generation tools whose definitions support Claude models (create_anthropic_message is 'Anthropic native... Supports Claude models'; create_chat_completion lists claude-sonnet-4-6 as an example model). A generic task like 'ask Claude this question' or 'have claude-sonnet-4-6 respond' is ambiguous across the two protocols; an agent may pick the wrong request format for the endpoint.
`transcribe_audio`	`translate_audio`	low	translate_audio's description reads 'Transcribes audio and translates the result to English', so a task like 'transcribe this audio' (without mentioning English translation) could plausibly match both definitions, leading an agent to pick the translating variant and produce English-only output instead of a true transcription.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard