01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 10%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.7 / 30
03Economics
19.9 / 20
04Discoverability
11.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_workflows |
name_restates_behavior no_return_description |
List the workflow definition files in a repository to discover which workflows, IDs, and file names exist before triggering or filtering runs. Returns workflow metadata including id, name, and file path. |
list_runs |
name_restates_behavior no_return_description |
List workflow runs for a repository, optionally narrowed to a specific workflow or status, to inspect execution history and outcomes. Returns runs with their IDs, status, conclusion, and timing details. |
get_run |
name_restates_behavior no_return_description |
Fetch detailed status information for a single workflow run by its run ID, such as its current status, conclusion, timing, and the workflow it belongs to. Returns the full run object for the specified run. |
rerun_workflow |
no_return_description |
Re-run all jobs of an existing finished workflow run, e.g. to retry the full pipeline after failures or changes. Returns confirmation that the re-run was queued. |
rerun_failed_jobs |
no_return_description |
Re-run only the failed jobs of a workflow run, leaving previously successful jobs untouched, to retry a pipeline efficiently. Returns confirmation that the re-run was queued. |
cancel_run |
name_restates_behavior no_return_description |
Stop a workflow run that is currently in progress or queued, used when the run is no longer needed or is stuck. Returns confirmation that cancellation was requested. |
list_artifacts |
name_restates_behavior |
List the artifacts produced by a specific workflow run, such as build outputs or reports uploaded during the run, to identify downloadable files. Returns artifact metadata including id, name, and size. |
trigger_workflow |
name_restates_behavior no_return_description |
Manually start a workflow run via workflow_dispatch on a specified branch or tag, for workflows that have workflow_dispatch enabled, to run the pipeline on demand. Returns confirmation that the workflow was triggered; poll with list_runs to track progress. |
Selection evidence
Confusable tool pairs.
4 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_run |
get_run_logs |
medium | Both share identical inputs (owner, repo, run_id) and overlapping names; a task like 'get the run 12345 and show what happened' or 'fetch the run output' is ambiguous between getting run details and fetching its logs. |
list_workflows |
list_runs |
medium | A task like 'list the workflows in this repo' is genuinely ambiguous because the agent must decide whether 'workflows' means the workflow definition files (list_workflows) or the workflow runs (list_runs); only descriptions disambiguate files vs runs. |
rerun_workflow |
trigger_workflow |
medium | Both start a workflow run and share the 'run workflow' tokens; a task like 'rerun the CI workflow' or 'run the deployment again' is ambiguous between re-running an existing run (needs run_id) and dispatching a fresh workflow run (needs workflow_id). |
rerun_workflow |
rerun_failed_jobs |
medium | Both 'rerun' a workflow run with identical inputs; a task like 'retry the workflow' or 'rerun the failed run' does not clearly state whether to re-run the whole workflow or only the failed jobs, so an agent could pick either. |
Compare the field