Public assessment

cnghockey/sats4ai-mcp-server (sats4ai-mcp)

sats4ai-mcp · v1.0.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 98→96, token cost down 2%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

21.2 / 30

21.2 out of 30

03Economics

5.4 / 20

5.4 out of 20

04Discoverability

9.3 / 20

9.3 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

18 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`analyze_image`	no_return_description	Analyze and describe image content, answer visual questions, and extract information from screenshots or photos. Uses Qwen VL — multimodal vision-language model with strong OCR, chart reading, and spatial reasoning. Returns JSON with the model's text answer/description of the image. 21 sats per image. Pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='analyze_image'.
`generate_text`	params_unexplained no_return_description	Generate text using frontier AI language models and return the result as JSON (the model's completion/reply). systemPrompt sets an optional custom instruction that shapes the model's behavior for this request. Pure per-character pricing (no minimum): Kimi K3 (best, ~10 chars/sat, 1M context, vision support, default), GPT-OSS-120B (standard, ~1000 chars/sat, 119 languages, best value). Rates are BTC-pegged and re-quoted hourly, so treat them as approximate — the 402 challenge is the authoritative price. Supports document Q&A via fileContext and vision analysis via imageBase64 (best model). Stable endpoints — models upgrade automatically. Pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='generate_text' and the exact prompt.
`extract_document`	no_return_description	Extract text from PDFs and images as clean Markdown. Uses Mistral OCR — handles complex layouts, tables, handwriting, multi-column documents, and mathematical notation, preserving document hierarchy in structured Markdown. Returns JSON containing the extracted Markdown text. 10 sats/page. Pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='extract_document' and quantity=pageCount for multi-page PDFs.
`send_email`	no_return_description	Reach anyone with an email address — useful when your task requires formal communication, sending reports, or contacting someone outside chat. No SMTP server, no domain verification needed. Plain text, max 10,000 chars body, 200 chars subject. Returns JSON confirming the email was sent (delivery status). 200 sats. Pay with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='send_email'.
`edit_image`	params_unexplained	Edit an image with natural language instructions. Uses Nano Banana 2 — understands context, handles object addition/removal, style transfer, and inpainting. Returns JSON with image URL. outputFormat selects the file type of the edited result ('jpg', 'png', or 'webp'; default jpg). Resolution-tiered pricing: 1K=200 sats, 2K=300 sats, 4K=450 sats. Pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='edit_image' and resolution param.
`translate_text`	no_return_description	Translate text across 119 languages with high accuracy using GPT-OSS 120B — top scorer in our multilingual translation evals with strong low-resource support. Auto-detects source language. Returns JSON with the translated text. Privacy-preserving: no data stored. Pricing: 1 sat per 1,000 characters, minimum 1 sat per request. Language parameters accept English names ('Spanish', 'Chinese (Simplified)') or ISO-639 codes / locale tags ('es', 'en-US', 'pt-BR', 'zh-Hans'). Pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='translate_text' and prompt (the text to translate).
`e_signature`	no_return_description	Send a document out for legally binding e-signature. Upload the PDF, name 1-3 signers, and give the email where the signed copy should land. Returns a submission confirmation immediately; the legally binding signed copy is delivered by email to customerEmail once all signers have completed signing (the signed PDF is not returned by this call — it arrives by email). Flat 1000 sats per document. The request is fulfilled through an established e-sign provider. Privacy: the PDF is processed in memory and never stored. Pay with Bitcoin Lightning — no account, no card. Requires create_payment with toolName='e_signature'.
`send_sms`	no_return_description	Reach a human via SMS when your task requires real-world coordination. Send to any phone number worldwide — messages delivered in seconds. Returns a confirmation that the message was accepted for delivery. No phone plan, no SIM card, no telecom account needed. Pay with Bitcoin Lightning — no API key, no KYC, no subscription. Requires create_payment with toolName='send_sms' and phoneNumber+message at payment time; the phoneNumber and message must match those used in create_payment.
`place_call`	no_return_description	Bridge the digital-physical gap — place an automated phone call to deliver a spoken message or play audio to any number. Useful when your task requires notifying a human, delivering alerts, or reaching someone who isn't online. Returns call placement confirmation and status for the call. Pay with Bitcoin Lightning — no telecom account, no KYC, no subscription. Requires create_payment with toolName='place_call' and phoneNumber.
`send_fax`	no_return_description	When your task requires a paper-trail on the other end — loan paperwork to a bank, signed contract to a notary, booking confirmation to a hotel in Japan — send a fax to any number worldwide. Returns a confirmation that the fax was accepted for transmission. Two modes: 'pdf' (fetch from public URL) or 'text' (we format typed text into a PDF locally). Text/cover support Latin (incl. Central European, Vietnamese), Greek, Cyrillic, Japanese, Korean, Chinese (Simplified + Traditional), Thai, Hindi, Georgian, Armenian, Amharic; RTL scripts (Arabic/Hebrew) and emoji are rejected BEFORE your payment is consumed. Optional cover page. Pricing: 500 sats for up to 10 pages, +50 sats per additional page. Max 350 pages / 50 MB. Pass 'pages' to create_payment as 'quantity' to get the right invoice. Pay with Bitcoin Lightning — no fax machine, no phone line, no telecom account.
`get_model_pricing`	name_restates_behavior no_return_description	Look up the cost of an AI model before paying. Pass the numeric modelId (the same ID used in create_payment or returned by list_models) to get that model's exact price in sats. Free to call — no payment required. Returns the model's per-call pricing details.
`voice_bridge_say`	no_return_description	Inject audio into an open Voice Bridge call. Two modes: (1) text — we synthesize via OmniVoice TTS in any of 602 languages; (2) audio_base64 + encoding — bring your own audio (mulaw_8000 or pcm_l16_16000 for MVP). STT is automatically muted while we inject, so the agent doesn't hear itself. Returns an acknowledgement once the audio has been queued into the session. No additional payment — covered by the session deposit.
`upscale_image`	no_return_description	Upscale images 2x or 4x with neural super-resolution. Uses Real-ESRGAN (ICCV 2021, PSNR 32.73dB on Set5 4x, 100M+ production runs). Recovers real detail from low-resolution images — not interpolation. Optional face enhancement. Returns the upscaled image as base64. Stable endpoint — model upgrades automatically as SOTA evolves. 5 sats per image, pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='upscale_image'.
`restore_face`	no_return_description	Restore blurry, damaged, or AI-generated faces to sharp, natural quality. Uses CodeFormer (NeurIPS 2022, state-of-the-art FID 32.65 on CelebA-Test). Adjustable fidelity — balance between quality enhancement and identity preservation. Also enhances background and upsamples. Returns the restored image as base64. Stable endpoint — model upgrades automatically as SOTA evolves. 25 sats per image, pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='restore_face'.
`detect_objects`	no_return_description	Detect and locate objects in an image by name. Grounding DINO (open-set detector, ECCV 2024) — describe what to find in natural language and receive bounding box coordinates with confidence scores. Returns structured detections: for each object found, a bounding box and confidence score. Structured pixel data agents can't get from vision LLMs. 5 sats per image, pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='detect_objects'.
`remove_object`	no_return_description	Remove unwanted objects from images by describing what to remove — no mask needed. Combines Grounding DINO detection (ECCV 2024) with Bria Eraser inpainting. Just say 'person', 'car', or 'watermark' and the object is erased and filled convincingly. Returns the resulting image with the object removed, as base64. 130 sats per image, pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='remove_object'.
`colorize_image`	no_return_description	Colorize black-and-white or grayscale photos. DDColor (dual-decoder, ICCV 2023) — vivid, natural colorization. Returns the colorized image as base64. Impossible for text/vision LLMs. 5 sats per image, pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='colorize_image'.
`deblur_image`	no_return_description	Recover detail from camera-shake and accidental motion blur. NAFNet (ECCV 2022, SOTA on GoPro/SIDD benchmarks). Best for: handheld shake, bumped camera, whole-frame uniform blur. NOT effective for: intentional panning blur, bokeh/depth-of-field, or artistic motion effects. Also supports denoising (grainy/noisy photos). Returns a JSON object containing the processed result, including the deblurred/denoised image (Base64-encoded or URL) and status info. 110 sats per image (~2 min processing), pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='deblur_image'.

Selection evidence

Confusable tool pairs.

12 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`ai_call`	`confirm_ai_call`	medium	A user task like 'confirm my appointment by calling the office' contains both 'confirm' and 'call', matching confirm_ai_call's name; an agent can pick confirm_ai_call (which only proceeds on an existing pending session) instead of ai_call, or pick ai_call when asked to continue a call that returned pending_confirm.
`check_payment_status`	`check_job_status`	medium	Identical check_*_status surface, both free and both returning a status/next-style object. A task phrased 'check the status of my request/order/video' without naming the entity type is ambiguous — checking the job when the user means the Lightning payment (or vice versa) returns misleading output.
`generate_image`	`generate_video`	medium	Both create visual media from a text prompt with the same paymentId+prompt+modelId interface. A task like 'generate a creative visual of X' or 'make a clip for the ad' doesn't pin down image vs video, and picking wrong spends a payment and returns the wrong media type.
`transcribe_audio`	`transcribe_translate`	high	Names differ only by '_translate' and both are async audio jobs. 'Transcribe this audio into English' or 'transcribe this Spanish voicemail' is genuinely ambiguous: transcribe_audio's language param is the source language while transcribe_translate uses targetLanguage, so an agent can pick transcribe_audio and get a garbled transcript when the user wanted translation.
`transcribe_translate`	`translate_text`	medium	A task like 'translate this recording/message to English' is ambiguous about input type: audio → transcribe_translate (STT + translation), text → translate_text (text only). An agent can route an audio file to translate_text, which only accepts text, or pick transcribe_translate for plain text input.
`extract_document`	`extract_receipt`	medium	Both take documentBase64 (PDF/image) and both 'extract' data. 'Extract the info from this invoice/receipt PDF' fits extract_document (Markdown text) just as well as extract_receipt (structured financial JSON); extract_receipt even warns that any text-bearing document is accepted, so input-type ambiguity is real.
`convert_file`	`convert_html_to_pdf`	medium	An HTML→PDF request matches both: convert_file is the generic 200+ format converter that includes PDF, while convert_html_to_pdf is purpose-built. The two descriptions even cross-reference each other, so a generic 'convert this HTML/page to a PDF' can pick the wrong one (and pricing differs: 100 vs 50 sats).
`get_job_result`	`await_result`	medium	Both retrieve async-job output with the same requestId/jobType inputs and near-identical output shapes. 'Get me the result of the video/transcription job' is ambiguous — get_job_result errors unless the job is already completed, while await_result waits and returns in one call, so picking get_job_result for a still-running job breaks the task.
`send_email`	`send_fax`	medium	Both deliver documents to an external party: 'formally send this report' → email, 'send the signed contract/loan paperwork' → fax. A task like 'send the signed paperwork to the bank' matches both descriptions, so the delivery channel the user wants is genuinely ambiguous.
`send_sms`	`send_fax`	medium	Both 'send' to an E.164 phone number and both fit confirmations/notifications ('send the booking confirmation to the hotel'). A task that doesn't name the channel (text vs fax) is ambiguous, and both tools take the same phoneNumber-based parameters.
`generate_text`	`translate_text`	medium	Translation can plausibly be done by either tool: translate_text is the dedicated 119-language translator, but generate_text is a general LLM whose default model also advertises 119 languages. A task phrased 'generate/translate the Spanish version of this text' is genuinely dual-route, with different pricing.
`poll_voice_bridge`	`end_voice_bridge`	medium	Both take only a sessionId and both surface transcript content (end_voice_bridge 'also returns the final transcript for convenience'). A task like 'get what the caller said' can make an agent call end_voice_bridge to fetch the transcript, prematurely hanging up and refunding the session instead of just polling.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard