01Safety
0.0 / 30
What changed in the harness
Selection accuracy 98→100, token cost up 1%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
27.2 / 30
03Economics
15.5 / 20
04Discoverability
12.0 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
3 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
vk_wall_create_comment |
params_unexplained |
Add a comment to a post on a wall. The post is addressed by owner_id (negative for a community, defaults to the token owner) together with the post_id of the post on that wall; provide the comment text in message. Returns the new comment_id. |
vk_friends_get |
params_unexplained |
List a user's friends, defaulting to the token owner. Returns bare user IDs unless fields is set, in which case it returns profile objects. Requires a user token and only works for profiles that expose their friend list. order selects the sort order (hints — relevance, random, or name) and count limits the number of friends returned. |
vk_stats_get |
params_unexplained |
Read a community's statistics — reach, visitors and activity — aggregated by period. interval selects the bucket size (day, week, month, year, or all to span the full history) and intervals_count limits how many such buckets to return. The token owner must be an administrator of the community, otherwise VK denies access. Returns one stats entry per period. |
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
vk_users_search |
vk_groups_search |
high | Both tools have nearly identical descriptions (find by keyword, narrowed by city/country, paginated) and only differ by entity type. A task like 'search for hiking near Berlin' or 'find football communities in Moscow' gives no cue whether the target is a person or a community, so the agent can easily pick the wrong entity type. |
vk_groups_get |
vk_groups_get_by_id |
medium | Shared 'get groups' wording: a task like 'get the group apiclub' or 'fetch my groups' maps to listing the token owner's communities in one tool and looking up a specific community by ID in the other. If the user doesn't say 'my' vs 'by name/ID', either pick is defensible from the names alone. |
vk_wall_get |
vk_wall_get_by_id |
medium | A task like 'get the post on this wall' is ambiguous: vk_wall_get reads a wall feed while vk_wall_get_by_id fetches specific posts by {owner_id}_{post_id}. Wording such as 'fetch the last wall post' could route an agent to the feed reader when the user wants a particular post, or vice versa when only knowing the wall. |
vk_users_get |
vk_users_search |
medium | Both resolve or locate people: vk_users_get 'looks up a user by numeric ID or short name', while vk_users_search 'finds users by name'. Tasks phrased as 'find the user durov' or 'get people named Ivan' are ambiguous between a direct name-lookup and a keyword search, so an agent can pick the wrong one based purely on wording. |
vk_users_get |
vk_groups_get_by_id |
medium | Both are near-identical 'look up {entity} by numeric ID or short name' tools. A short name like 'durov' or 'apiclub' could plausibly be a person or a community, so a task such as 'resolve the profile apiclub' or 'get info about durov' gives the agent no reliable signal about entity type without extra context. |
vk_wall_post |
vk_photos_upload_wall |
low | For a task like 'post this photo to the group wall', the agent must first upload via vk_photos_upload_wall and then publish via vk_wall_post. Both tools look like plausible starting points for 'posting' an image, and an agent could call vk_wall_post directly with the raw image or stop at upload thinking the photo was published. |
Compare the field