Public assessment

uxloom-dev/uxloom (uxloom)

uxloom · v0.5.0 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 93→95, token cost down 0%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

24.7 / 30

24.7 out of 30

03Economics

14.2 / 20

14.2 out of 20

04Discoverability

17.2 / 20

17.2 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

7 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`project_init`	no_return_description	Create a new UXLoom project file (uxloom.project.json) in the current directory, seeded with the given product name and target platforms. Run once per product. Returns a confirmation that the project file was created (or an error if one already exists).
`brief_answer`	no_return_description	Submit answers to a brief's questionnaire (matched to the same 'prompt' used in brief_start) so the design brief can be finalized. Unanswered fields take researched defaults and are recorded in the assumption ledger (auditable, reversible). Returns the finalized brief along with the list of applied default assumptions.
`journey_define`	no_return_description	Add or replace a journey (a state machine: states reference screens, events move between states) in the project. Screens referenced here must be registered via screen_register before project_validate passes. Returns a save confirmation plus any immediate validation errors (e.g. references to unregistered screens).
`screen_register`	no_return_description	Add or replace a screen: its intent, requiredStates (the contract), designedStates (progress), and components with colors/labels/target sizes for the critics. Returns a save confirmation plus any immediate validation errors.
`project_import`	no_return_description	Replace the whole project in one call: journeys and screens together. Prefer this over many journey_define/screen_register calls when registering a complete or large design. Validates the full document; unknown fields are rejected. Returns a save confirmation along with any validation errors found across the imported journeys and screens.
`palette_check`	params_unexplained	Check a design system's color pairs against WCAG 2.2 AA (4.5:1) contrast requirements before any screens exist. 'pairs' is a list of {name, fg, bg} entries, each a human-readable name plus a foreground and background hex color to test together. Reports each pair's exact ratio, pass/fail, and thin-margin passes (under 5.0:1) that one shade lighter would break.
`screen_critique`	params_unexplained no_return_description	Return validation findings, with fixes, scoped to a single screen identified by screenId (the id used when the screen was registered via screen_register). Use during iteration on one screen instead of running a full project_validate.

Selection evidence

Confusable tool pairs.

12 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`project_audit`	`project_validate`	high	Both are whole-project check tools returning findings/fixes; a task like 'check my project for issues' could plausibly invoke either, though audit targets implementation drift vs validate targets design-contract critics.
`project_export`	`project_audit`	low	Distinct purposes (return document vs audit implementation) with little verb overlap; unlikely to be confused.
`project_export`	`project_validate`	low	Export retrieves data while validate runs critics; task phrasing rarely overlaps.
`project_import`	`project_audit`	low	Import writes/replaces project data while audit reads implementation state; minimal functional overlap.
`project_import`	`project_export`	medium	Both operate on the whole project document (one reads, one writes) and a vague task like 'save/update my project' could momentarily conflict, though export has no input and import requires the full document, reducing real confusion.
`project_import`	`project_validate`	low	Import replaces project content; validate only runs critics with no input—different actions unlikely to be swapped.
`project_init`	`project_audit`	low	Init creates a brand-new project file while audit checks an existing implementation; little ambiguity.
`project_init`	`project_export`	low	Init creates a new project; export retrieves an existing one—opposite directions, unlikely confusion.
`project_init`	`project_import`	medium	Both can be seen as 'starting' a project; a task like 'set up my project with these screens and journeys' could plausibly be routed to either init (name/platforms only) or import (full document), especially if the agent doesn't realize init is for empty projects only.
`project_init`	`project_validate`	low	Init creates the project file; validate audits an existing one for errors—clearly different purposes.
`screen_register`	`screen_critique`	low	Register adds/updates a screen definition while critique returns findings for a screen; verbs differ enough ('add' vs 'review') that confusion is unlikely despite shared 'screen' scope.
`brief_start`	`brief_answer`	low	These are sequential steps in one workflow (start then answer) rather than alternatives for the same task, so an agent naturally calls both in order rather than choosing one over the other.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard