Public assessment

argosvix/mcp-server (@argosvix/mcp-server)

argosvix-mcp-server · v1.1.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 95→93, token cost up 3%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

20.2 / 30

20.2 out of 30

03Economics

6.5 / 20

6.5 out of 20

04Discoverability

11.5 / 20

11.5 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

37 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`query_calls`	no_return_description	Retrieve recent LLM call records captured by Argosvix, newest-first. Returns a list of matching records — each with fields such as id, timestamp, provider, model, response latency (ms), and tags — up to the limit (default 100, max 500) over the selected time range (default last 24h). Filterable by provider / model / latency bounds / tag (tagKey + tagValue pair) and paginated via the beforeTimestamp + beforeId keyset cursors.
`reply_proposal`	no_return_description	Post a question about a proposal and receive the AI's reply as it would appear in the inbox conversation. Returns the AI's reply (thread text) for the given proposalId and body. Explanation only — nothing is executed. Get proposalId from list_proposals.
`silence_alert`	no_return_description	Temporarily mute an alert so notification delivery stops until the effective expiry. Returns the silenced alert id and the effective mute-until timestamp. Defaults to 24 hours from now; pass an ISO-8601 timestamp as until for a custom expiry. Pass the alertId obtained from list_alerts.
`unsilence_alert`	no_return_description	Unmute a currently silenced alert so its notifications resume. Returns a confirmation (with the unmuted alertId) that silencing is no longer active. Pass the alertId obtained from list_alerts.
`update_alert`	no_return_description	Update an existing alert's settings (PATCH /v1/alerts/:id) and return the updated alert configuration (including its id). alertType (the watched metric type) is immutable — to change it, create a new alert and then delete the old one (completing the alert lifecycle). Threshold / evaluation window / notification channels / name / enabled flag / composite conditions can be partially updated (all fields optional). Example phrasing: "lower the monthly budget alert threshold from $100 to $50" / "add Slack as a notification channel".
`delete_alert`	no_return_description	Delete an alert (DELETE /v1/alerts/:id) and return a deletion confirmation for the given alertId. Related alert_events are CASCADE-deleted too. To guard against accidental deletion, checking the details with get_alert first is recommended. If you only want to pause an alert, prefer silence_alert (mute) or update_alert with enabled=false instead of delete (both are recoverable).
`create_annotation`	no_return_description	Create a new annotation (human review / labeling) for an LLM call and return the created annotation (with its id and fields annotationText / label / qualityScore). Specify at least one of annotationText / label / qualityScore (an empty annotation gets 400 from the backend). Example phrasing: "Claude, label this call 'badly-summarized' with quality 2", or bulk-apply positive / negative labels for an eval loop. Combined with the eval baseline runner (run_eval), annotations can calibrate eval criteria as ground truth.
`update_annotation`	no_return_description	Partially update an annotation's annotationText / label / qualityScore (PATCH /v1/annotations/:id) and return the updated annotation. callId is immutable. For fixing a label or re-scoring quality from 4 to 5, etc. Pass annotations[].id obtained from list_annotations_for_call as annotationId.
`delete_annotation`	no_return_description	Delete an annotation (DELETE /v1/annotations/:id) and return a deletion confirmation. No other rows depend on it, so there is no CASCADE impact. To guard against accidental deletion, checking the details with get_annotation first is recommended. Pass the annotationId obtained from list_annotations_*.
`create_eval_criterion`	no_return_description	Create one custom eval criterion in your account (Pro+ only) and return the created criterion (including its id). name + rubric + scaleMin + scaleMax are required. Same name already existing in the account = 409. A name matching a global default is structurally allowed (UNIQUE (account_id, name) separates it from account_id IS NULL). type defaults to 'llm_judge' (judge LLM scoring). Specifying a deterministic evaluator type (exact_match / contains / regex / json_schema / json_path) scores without calling an LLM — free and instant (pass -> scaleMax / fail -> scaleMin). Deterministic types require config. The path an AI agent takes when it decides "add this criterion" while evaluating your own workloads.
`update_eval_criterion`	no_return_description	Update a custom criterion in your account with a full replace (Pro+ only, PATCH /v1/eval-criteria/:id) and return the updated criterion. name + rubric + scaleMin + scaleMax are required (not a partial update — all fields are overwritten). type / config are also fully replaced (omitting them reverts to 'llm_judge' / no config). Deterministic types require config. Global defaults (account_id IS NULL) are structurally out of scope (404); other accounts' customs are 404 too. Name collision within the account = 409.
`raise_llm_budget`	no_return_description	Raise or lower the monthly LLM feature budget (Pro+ only). Range $5 - $500 (hard cap against runaway spend), in $0.01 increments. Existing spend carries over; auto-resets at month boundaries. A new value below current spend is accepted (remaining simply becomes 0; counting restarts from 0 next month). Examples: we hit 80% of the budget, so raise it to $30 just for this month; or we overspent, so lower next month to $10. Applies the new limit immediately and returns a success confirmation; spend already incurred carries over into the new budget.
`create_budget_gate`	no_return_description	Create a runtime budget gate (Pro+ only). Sets a monthly LLM spend limit (USD) for the account; the SDK (budgetGate opt-in) blocks over-limit calls before execution. Enforcement is optimistic (spend is cached for 60 seconds and in-flight calls pass, so the limit is a guideline that can be exceeded, not a strict hard cap). enforceMode = fail_open (default; calls pass when the backend is unreachable) / fail_closed (calls are blocked when unreachable; a cold start where the SDK has never fetched the config additionally requires the SDK-side failClosed opt-in). Omitting projectId creates an account-wide gate (only one; 409 if one exists). Specifying projectId creates a gate for that project only (ANDed with the account gate — the strictest limit wins; one per project). Specifying tagKey plus tagValue creates a gate for calls carrying that tag (e.g. tagKey=service / tagValue=checkout caps the monthly spend of service=checkout; ANDed with the account gate; one per tagKey/tagValue pair). tagKey/tagValue must be specified together and are mutually exclusive with projectId. Example phrasing: create a budget gate at $50/month; cap project X at $10/month; cap the service=checkout tag at $20/month. On success returns the created gate (bg_-prefixed id and settings, same shape as the get_budget_gate Response).
`update_budget_gate`	no_return_description	Update a runtime budget gate (Pro+ only). Partially updates any of monthlyLimitUsd / enforceMode / enabled. Example phrasing: raise the limit to $100; disable the gate temporarily; switch to fail_closed. Applies only the provided fields and returns the updated gate (same shape as the get_budget_gate Response).
`delete_budget_gate`	no_return_description	Delete a runtime budget gate (Pro+ only). After deletion the SDK's pre-execution enforcement is disabled. To pause temporarily, prefer update_budget_gate with enabled: false. On success the gate is removed (it no longer appears in get_budget_gate) and the call returns a success response.
`get_approval`	no_return_description	Get the current state of an approval request. Returns the approval record, including its status (pending / approved / denied / expired) together with the approvalId and the action/summary/expiry from the original request_approval call. Do not perform the target operation unless the status is approved (default-deny). Dangerous mutation tools also support server-side consumption via their approvalId param (see the request_approval description).
`list_approvals`	name_restates_behavior no_return_description	Fetch the most recent approval requests (latest 50). Filter by status: pending (default) / approved / denied / expired / all. Returns the matching approval records, each with the same shape as get_approval (approvalId, action, summary, status, expiry, etc.) — use the returned approvalId to poll a request with get_approval, and do not act until a request is approved.
`create_policy_gate`	no_return_description	Create a runtime policy gate (Pro+ only). Configures an account-wide model allowlist / PII block / secret block; the SDK (policyGate opt-in) blocks violating calls before execution. At least one rule (modelAllowlist / blockPii / blockSecrets) is required. One per account (409 if one exists). A redact mode is not supported (block only). Example phrasing: only allow gpt-5.5 and claude-fable-5; block calls containing PII. On success returns the created gate (pg_-prefixed id and settings, same shape as the get_policy_gate Response).
`update_policy_gate`	params_unexplained no_return_description	Update a runtime policy gate (Pro+ only). Partially updates modelAllowlist (null clears the restriction) / blockPii (block on PII detection: emails, Luhn-verified card numbers, delimited phone and national ID numbers, IPv4 and IPv6 — undelimited digit runs excluded to avoid false blocking) / blockSecrets (block on API-key / private-key-like token detection) / enforceMode (fail_open passes calls when the backend is unreachable; fail_closed blocks them) / enabled (whether the gate applies). Example phrasing: add gpt-4o-mini to the allowlist; enable secret blocking. Applies the provided fields and returns the updated gate (same shape as the get_policy_gate Response).
`delete_policy_gate`	no_return_description	Delete a runtime policy gate (Pro+ only). To pause temporarily, prefer update_policy_gate with enabled: false. On success the gate is removed (no longer returned by get_policy_gate), the response confirms the deletion, and the SDK stops evaluating it against LLM calls.
`delete_eval_criterion`	no_return_description	Delete a custom criterion in your account (Pro+ only, DELETE /v1/eval-criteria/:id). Succeeds with HTTP 204 — the criterion and all its past eval_run score rows (eval_scores) are physically deleted at the same time via ON DELETE CASCADE, so historical comparisons and score trend analysis become permanently impossible. Global defaults (account_id IS NULL) and other accounts' criteria are out of scope and return 404. This is not a tool for an AI agent to call casually while tidying up criteria; only proceed when the user has explicitly confirmed the past run scores are not needed. If you only want to rename, using update_eval_criterion (full replace) with name + rubric + scaleMin + scaleMax preserves the history.
`create_webhook`	no_return_description	Register one outbound event webhook (Pro+ only, POST /v1/webhooks). url (HTTPS required; SSRF defense rejects private/loopback) + optional secret (HMAC-SHA256 signing key) + eventTypes (array of event kinds to subscribe to; omitted / empty = subscribe to everything). Up to 10 per account. Delivery payload = { event, eventId, occurredAt, accountId, data }; with a secret set, an X-Argosvix-Signature header is attached. On success returns the created webhook (owh_-prefixed id and settings — see list_webhooks for the returned fields).
`list_prompts`	no_return_description	List the prompt templates the user has registered. Returns up to limit prompt entries (default 200), each including id / name / version / template / variables / labels / description / createdAt, sorted by name ASC + created_at DESC. Filter by a label such as production (?label=xxx), or fetch all versions of one name (?name=xxx). This is the main path for an AI agent to read and use prompts the user registered in the dashboard.
`update_prompt`	no_return_description	Partially update an existing prompt's template / variables / labels / description (Pro+ only, PATCH /v1/prompts/:id). name + version are immutable (change them via rename_prompt). promptId is required; only the fields you pass are updated. Used by AI agents for label moves (promoting staging to production) and small patch edits. Applies the patch and returns the updated prompt record (same field set as list_prompts).
`rename_prompt`	no_return_description	Change an existing prompt's name + version (Pro+ only, POST /v1/prompts/:id/rename). Main use is typo fixes (customer_supprt to customer_support). Collision with an existing (name, version) in the account = 409. Since update_prompt never changes name/version by contract, rename is a separate tool for semantic separation. On success returns the updated prompt with the new name and version.
`rollback_prompt`	no_return_description	Revert the prompt deployed to a label to the previous version (Pro+ only, POST /v1/prompts/deployments/rollback). Returns the resulting deployment state, since after reverting another rollback toggles back (current / previous swap). 409 when there is no previous version (first deployment only), and 409 when the previous version has already been deleted.
`get_percentiles`	no_return_description	Get percentile metrics over calls (POST /v1/query/percentiles). Returns a single percentile value for the whole range, or a time series of values bucketed by 'day'/'hour'/'minute' when groupBy is set. metric = 'latency' (ms) or 'cost' (USD); percentiles are computed with the nearest-rank method. Example phrasing: "daily p95 latency trend for last week".
`list_projects`	name_restates_behavior no_return_description	Return the account's active projects (GET /v1/projects; archived excluded) so you can observe per-environment setups such as dev / staging / prod; includes each project's id (usable with rename_project / delete_project) plus its metadata. Pro allows 5 projects / Team unlimited; Free has the default project only.
`create_project`	no_return_description	Create a new project (POST /v1/projects) and return the created project including its id so it can be referenced by rename_project / delete_project. name = display name; slug = a short URL-safe identifier (/^[a-z][a-z0-9-]{0,31}$/). Pro caps at 5 projects, Team unlimited, Free cannot create (403). As a mutation, session-authenticated requests enforce Origin/Referer (dashboard-driven).
`rename_project`	no_return_description	Update an existing project's name / slug (PATCH /v1/projects/:id) and return the updated project. Specify either or both. slug keeps the URL-safe constraint (/^[a-z][a-z0-9-]{0,31}$/). Renaming the default project is allowed.
`delete_project`	no_return_description	Soft-delete a project (DELETE /v1/projects/:id; sets archived_at for a logical delete) and return the archived project (including archived_at). The default project cannot be deleted (400, keeping accounts.default_project_id referentially consistent). After archiving, calls / alerts remain as-is (past observations are kept); route new records to another project.
`list_audit_log`	no_return_description	List the audit log (GET /v1/audit-log), scoped to your account, admin role only (viewer/member get 403). Returns recent entries (each with event type, target kind, actor user id, and timestamps) covering invitations / API key revocations / project changes, plus a nextCursor when more results remain. Filters = eventType / targetKind / actorUserId / from / to; max limit 200.
`list_saved_views`	no_return_description	List the saved views for the account (GET /v1/saved-views). Returns an array of saved view objects, each with an id, name, and its stored filter (startDate/endDate/provider/model/limit); a preset default-filter view may also be present. Per account, max 20. Enables phrasing like "show calls with my usual last-week OpenAI filter" by retrieving the previously saved filter combination.
`create_saved_view`	no_return_description	Create a new saved view, or overwrite when the name exists (POST /v1/saved-views). name is unique within the account. filter follows the SavedViewFilter shape (startDate / endDate / provider / model / limit / preset / sortBy? / sortOrder?). Returns the created (or overwritten) saved view including its id, which can then be passed to the "show calls with a saved view" workflow. Lets an AI agent save frequently used filters under a name — e.g. create a "last 7 days, GPT-4 only" view and recall it later.
`delete_saved_view`	name_restates_behavior no_return_description	Delete the saved view identified by id (DELETE /v1/saved-views/:id); the id is a UUID from list_saved_views. Scoped to your account, so views belonging to other accounts are unaffected. Returns a success confirmation indicating the saved view was removed.
`get_eval_dataset`	no_return_description	Fetch one dataset's detail plus all of its items (GET /v1/eval-datasets/:id). datasetId is list_eval_datasets.datasets[].id. Returns the dataset metadata (name / description / item count / frozen state) together with an items array containing each test case, so the caller can see the exact expected outputs a population run will score against.
`delete_eval_dataset`	no_return_description	Delete a golden dataset (DELETE /v1/eval-datasets/:id, Pro+ only). Items are cascade-deleted. Past eval runs / scores remain. Returns a success confirmation that the dataset was deleted.

Selection evidence

Confusable tool pairs.

5 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get_prompt`	`get_deployed_prompt`	high	Both return a prompt's template/variables by name-ish reference, and the deployed-prompt description explicitly frames itself as 'the main runtime path for an agent to fetch the production prompt.' A task like 'get the production prompt' or 'fetch the current prompt for prod' could pick get_prompt (or list_prompts) instead of get_deployed_prompt, and the fact that get_prompt is keyed by registry ID while the natural task only names a label/name makes selection genuinely ambiguous.
`list_alerts`	`list_alert_events`	medium	Both describe alert triggers; list_alerts includes 'trigger history within the last 24 hours' and list_alert_events returns 'alert trigger events.' A task like 'which alerts fired recently?' or 'when did the cost alert go off?' maps to both descriptions, and an agent could pick list_alerts with includeTriggered=true or list_alert_events without clearly knowing which yields the firing history. The descriptions do differentiate (configured alerts vs account-wide events), mitigating but not eliminating the confusion.
`silence_alert`	`auto_silence_noisy_alert`	medium	auto_silence_noisy_alert accepts a single alertId and finishes 'silence it for an hour,' which heavily overlaps silence_alert's core purpose; nothing in the descriptions tells an agent to prefer silence_alert over auto_silence_noisy_alert for a plain single-alert silence. A task like 'mute alert X for 24h' could plausibly trigger either tool, though silence_alert's explicit alertId+until signature and simpler description give it a slight edge.
`list_prompts`	`list_prompt_deployments`	low	Both operate on prompts and a task like 'list my prompts' or 'what's deployed for prompt X' could hit either, but list_prompts clearly targets registered template versions while list_prompt_deployments explicitly describes 'current deployment states' — the descriptions and example use cases make the split reasonably clear, so confusion is only weakly plausible.
`run_eval`	`run_eval_dataset`	medium	Both have the verb 'run' plus 'eval' and trigger an eval scoring pass. A task like 'run an evaluation against our latest calls' or 're-run the eval and compare' could pick run_eval, while run_eval_dataset is the intended mechanism for regression A/B via golden datasets. However, run_eval_dataset's description sharply distinguishes the golden-dataset workflow (target model, datasetId), so ambiguity mostly arises when a task references 'eval run' without mentioning datasets — still a plausible mis-selection.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard