01Safety
0.0 / 30
What changed in the harness
Selection accuracy 93→95, token cost up 30%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
23.8 / 30
03Economics
20.0 / 20
04Discoverability
12.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
gist_list |
name_restates_behavior params_unexplained no_return_description |
Return a paginated list of the Gists owned by your GitHub account, ordered from newest to oldest. perPage controls how many gists are returned per page (1-100, default 10); page selects which page of results to fetch, starting at 1. The response is a JSON array of Gist objects, each containing the gist's id, description, file names, and URL. |
gist_get |
name_restates_behavior params_unexplained no_return_description |
Fetch a single existing Gist by its unique identifier. gistId is the ID string that appears in the gist's URL (for example 'abcdef123456'). The response includes the gist's metadata (description, owner, timestamps) plus the full contents of every file it contains, keyed by filename. |
gist_create |
name_restates_behavior no_return_description |
Create a new Gist under your GitHub account. files is a required map of filename to raw file content and determines which files the gist contains; description adds an optional title or note; public selects whether the gist is publicly visible (true) or secret/unlisted (false, default). Returns the created Gist object including its id, URL, and file contents. |
gist_update |
name_restates_behavior no_return_description |
Modify an existing Gist, identified by gistId, by updating its description and/or its files. Set a filename to a string to overwrite that file's content, set a filename to null to delete that file, and leave omitted properties unchanged. Returns the updated Gist object with its id, URL, and current file contents. |
gist_delete |
name_restates_behavior no_return_description |
Permanently remove an existing Gist from your account, identified by its gistId (the ID from the gist's URL). This action is irreversible and the gist's contents cannot be recovered. Returns a success response, after which the gist no longer appears in your gist list. |
gist_starred |
params_unexplained no_return_description |
Return a paginated list of the Gists you have starred on GitHub. perPage controls how many gists are returned per page (1-100, default 10); page selects which page of results to fetch, starting at 1. The response is a JSON array of Gist objects, each containing the gist's id, description, file names, and URL. |
gist_star |
name_restates_behavior no_return_description |
Mark an existing Gist, identified by its gistId, as starred in your GitHub account, the same action as clicking the star button on the gist's page. Starring is idempotent. Returns a success response with no content. |
gist_unstar |
name_restates_behavior no_return_description |
Remove the star from an existing Gist, identified by its gistId, in your GitHub account, the same action as unstarring on the gist's page. Unstarring is idempotent. Returns a success response with no content. |
Selection evidence
Confusable tool pairs.
2 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
gist_list |
gist_starred |
medium | Both are list operations with identical page/perPage schemas and near-identical descriptions differing only by 'starred'; users often conflate the gists they own with the ones they've starred, so a task like 'show me my gists' or 'list the gists I follow' can route to the wrong endpoint. |
gist_delete |
gist_unstar |
low | Both take a gistId and can be phrased with 'remove/drop'; a request like 'remove this gist from my starred list' or 'take this gist off' could trigger gist_delete instead of gist_unstar, though genuine deletion usually requires explicit intent. |
Compare the field