01Safety
0.0 / 30
What changed in the harness
Selection accuracy 95→91, token cost up 8%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
24.7 / 30
03Economics
19.9 / 20
04Discoverability
11.3 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_workflows |
name_restates_behavior no_return_description |
Retrieves the workflow definition files configured in the repository (e.g. files under .github/workflows/), returning each workflow's ID, name, file path, and state. |
list_runs |
name_restates_behavior no_return_description |
Retrieves recent workflow run history for the repository, optionally narrowed to a specific workflow_id or status, returning run metadata such as run ID, status, conclusion, branch, and trigger event. |
get_run |
name_restates_behavior no_return_description |
Retrieves full metadata for a single workflow run identified by run_id, returning fields such as status, conclusion, triggering event, branch, commit SHA, and timestamps. |
rerun_workflow |
no_return_description |
Re-runs every job in a previously executed workflow run (all jobs, not just failed ones), returning confirmation that the new run was queued. |
rerun_failed_jobs |
no_return_description |
Re-runs only the jobs that failed in a previous workflow run, leaving successful jobs untouched, and returns confirmation that the rerun was queued. |
cancel_run |
no_return_description |
Cancels a workflow run that is currently in progress or queued, returning confirmation that the cancellation request was accepted. |
list_artifacts |
name_restates_behavior no_return_description |
Retrieves the build artifacts uploaded during a specific workflow run, returning each artifact's name, size, expiration date, and download URL. |
trigger_workflow |
no_return_description |
Starts a new run of a workflow that has a workflow_dispatch trigger defined, using the given ref and optional inputs, and returns confirmation that the run was queued. |
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_run |
get_run_logs |
medium | A request like "show me what happened in run 123" could plausibly be served by either fetching run metadata or fetching logs; agent may pick get_run when logs are wanted or vice versa. |
rerun_workflow |
rerun_failed_jobs |
high | "Re-run the failed CI run" is ambiguous between re-running the entire workflow and re-running only the failed jobs; both take identical params and differ only in description nuance. |
list_workflows |
list_runs |
medium | "List the workflows for this repo" could be misread as wanting run history; both are repo-level list operations differing mainly in whether they list workflow definitions vs run instances. |
rerun_workflow |
trigger_workflow |
medium | "Run the workflow again" could be interpreted as re-running a prior run (rerun_workflow, needs run_id) or dispatching a fresh run (trigger_workflow, needs workflow_id/ref), especially if the user doesn't clearly specify which. |
get_run |
cancel_run |
low | Both operate on a run_id and share 'get the run' phrasing in casual requests like 'stop/check run 123', but their action verbs (get vs cancel) are distinct enough to rarely confuse. |
list_runs |
list_artifacts |
low | Both are list operations tied to workflow runs, but 'list runs' vs 'list artifacts from a run' target different entities and are unlikely to be swapped given clear schema differences (owner/repo vs run_id). |
Compare the field