01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost down 0%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
28.4 / 30
03Economics
14.6 / 20
04Discoverability
11.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
2 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
vibekit_db_query |
params_unexplained |
Run a read-only SELECT query against an app's Postgres database and return up to 200 result rows. The sql parameter is a single SQL string, max 5000 chars, and is strictly read-only — writes and DDL (INSERT/UPDATE/DELETE/ALTER/DROP/…) are rejected server-side; use vibekit_chat or vibekit_submit_task to have the agent make data or schema changes. Call vibekit_db_schema first to learn the tables. |
vibekit_get_task |
name_restates_behavior |
Check on a task you submitted with vibekit_submit_task by its taskId. Returns the task's current status (e.g. pending, running, completed, failed) plus a result/logs summary and repo info once progress is available; call repeatedly to poll progress instead of blocking. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
vibekit_db_query |
vibekit_db_table |
medium | Both are read-only ways to pull table data; a request like 'show me the users in my database' or 'browse the orders table' plausibly maps to either, since db_table is explicitly described as a quick alternative to hand-writing a SELECT. |
vibekit_get_task |
vibekit_list_tasks |
medium | Both surface task status; a natural query like 'what's the status of my task?' or 'did my tasks finish?' could route to the wrong member because get_task needs a taskId while list_tasks filters by status, so a status request without an ID may trigger get_task with an assumed ID. |
vibekit_agent_status |
vibekit_agent_history |
medium | Both return recent output of the same app's AI agent, so 'get the agent's latest reply/update' is ambiguous: status carries live activity progress and a liveCheck while history holds the final chat messages, and an agent may pick either for a generic 'what did the agent do?' request. |
vibekit_app_env |
vibekit_set_env |
medium | Both act on an app's environment variables with the same appId, so phrasing like 'configure/update my app's environment' or 'check my env vars' is ambiguous without an explicit read/write verb; the read-only description reduces but does not eliminate the get/set confusion. |
Compare the field