01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→98, token cost down 0%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.6 / 30
03Economics
14.3 / 20
04Discoverability
10.8 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
vibekit_get_task |
name_restates_behavior |
Look up a task previously submitted via vibekit_submit_task by its taskId and return its current lifecycle state (pending/running/completed/failed) plus logs/result summary and repo info. Use this for a single point-in-time check; use vibekit_wait_for_task instead if you want to block until it finishes. |
Selection evidence
Confusable tool pairs.
9 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
vibekit_db_query |
vibekit_db_table |
high | A request like 'show me the rows in the orders table' or 'browse the users data' could plausibly be served by either a hand-written SELECT via db_query or the paginated browse via db_table — both are read-only row-fetching tools over the same data with overlapping use cases. |
vibekit_db_schema |
vibekit_db_table |
medium | A vague ask like 'what does the users table look like' is ambiguous between wanting column/structure info (db_schema) and wanting to see actual row contents (db_table), especially since both take just appId+table-like context. |
vibekit_agent_status |
vibekit_agent_history |
medium | 'What did the agent do to my app' or 'what's happening with my build' could route to either the live progress/status tool or the past chat-history tool, since both surface information about the same agent activity. |
vibekit_deploy |
vibekit_chat |
medium | A task like 'deploy the latest version of my app' overlaps: vibekit_chat has the AI agent edit code and redeploy, while vibekit_deploy ships a GitHub repo fresh — an agent unsure whether the app already exists could pick the wrong one. |
vibekit_app_env |
vibekit_set_env |
medium | A task phrased as 'update/configure the API key for my app' could be misrouted to the read-only vibekit_app_env (which only gets env vars) instead of vibekit_set_env, since both tools are tightly coupled around 'app env vars'. |
vibekit_run_qa |
vibekit_qa_status |
medium | 'Check the QA on my app' is ambiguous between kicking off a new automated QA run (vibekit_run_qa) and retrieving the latest existing results (vibekit_qa_status). |
vibekit_get_task |
vibekit_list_tasks |
low | 'Check on my task' without an explicit taskId could tempt an agent to call list_tasks instead of get_task, though the required taskId parameter on get_task makes the distinction fairly discoverable. |
vibekit_restart_app |
vibekit_stop_app |
low | Both act on a running container's lifecycle ('take my app offline for a bit') and share strong token overlap, though the verbs 'restart' vs 'stop' are usually unambiguous in natural phrasing. |
vibekit_list_skills |
vibekit_get_skill |
low | 'Get me the skill for auth' could be attempted directly via get_skill without first listing, but get_skill's required id parameter (sourced from list_skills) makes the intended sequence fairly clear from the schema. |
Compare the field