01Safety
0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 7%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
26.8 / 30
03Economics
20.0 / 20
04Discoverability
11.6 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
5 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
asc_doctor |
no_return_description |
Run this first in any release session to verify the release toolchain end-to-end: check the ASC API credentials, the mac-app directory, fastlane, xcodebuild, and (if creds are present) that the app record exists in App Store Connect. Returns a report of which checks passed or failed along with any error details. |
asc_app_status |
no_return_description |
Returns a snapshot of the app's release state: the latest App Store versions with their review states, and recent builds with their processing states. Use this to answer 'where is the release right now?' |
asc_list_builds |
params_unexplained no_return_description |
List recent uploaded builds and their processing states (PROCESSING → VALID before they can be submitted). Use after an upload to watch for the build becoming VALID. Returns the builds with their IDs and processing states; limit controls how many recent builds are returned (default 10, max 50). |
asc_bump_version |
no_return_description |
Bump the local app version before a build. Increments the build number (CURRENT_PROJECT_VERSION) and optionally sets a new marketing version. Updates every .xcodeproj/project.pbxproj in the project dir, plus project.yml if the project uses xcodegen, so all sources stay in sync. Returns the new build number and marketing version so you can confirm the change. |
asc_update_metadata |
params_unexplained no_return_description |
Update App Store listing metadata (description, keywords, what's-new, promotional text) on the currently editable version via the ASC API. If no editable version exists, pass create_version to open a new one. `locale` selects the language of the version to edit (default 'en-US'); `description` sets the App Store listing description; `promotional_text` sets the optional promotional text shown above the description. Returns the updated version metadata, or an error if no editable version exists and create_version is not provided. |
Selection evidence
Confusable tool pairs.
6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
asc_app_status |
asc_list_builds |
high | Both descriptions surface 'recent builds and their processing states'; a task like 'list recent builds and their processing states' or 'where is my release right now' matches both nearly identically, so a wrong pick is very plausible. |
asc_app_status |
asc_job_status |
medium | Both are 'status' style checks involving builds; a task like 'check the status of my build/release' could surface either, and an agent may call asc_job_status without a job_id or asc_app_status when only the upload job state was meant. |
asc_bump_version |
asc_update_metadata |
medium | Both let the agent 'set a version' on the app: bump_version accepts a marketing_version, while update_metadata has create_version; a task like 'set the version to 1.1.0' or 'update the app's version' is ambiguous between bumping the local project and creating/updating the App Store listing version. |
asc_app_status |
asc_submit_review |
medium | Both involve 'review' of the app: app_status reports review states and submit_review submits for review; a task like 'handle the review for my release' or 'what's happening with my review' could be read as checking the review status rather than submitting, leading to the wrong member. |
asc_list_builds |
asc_upload_build |
low | Both contain 'build' and 'upload'; a multi-intent task like 'make sure my build is uploaded and shows up' risks listing builds (asc_list_builds is the check-after-upload tool) when the user wanted a fresh upload, though the primary intent ('upload' vs 'list') usually disambiguates. |
asc_doctor |
asc_app_status |
low | A vague task like 'check my release' or 'is my app ready to release' could target doctor (toolchain/credential readiness) or app_status (release state); the shared app/release/store tokens make this plausible, though doctor's 'run first' framing and status-name reduce real confusion. |
Compare the field