Public assessment

Sowiedu/Edict (edict-lang)

edict-lang · v1.23.2 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 96→89, token cost up 4%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

24.8 / 30

24.8 out of 30

03Economics

17.6 / 20

17.6 out of 20

04Discoverability

11.0 / 20

11.0 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

9 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`edict_version`	no_return_description	Return the version of the edict-compiler server and the Edict language/compiler version it implements.
`edict_examples`	no_return_description	Return a set of example Edict programs demonstrating the language's features, useful for learning the AST format and common patterns.
`edict_validate`	name_restates_behavior no_return_description	Validate an Edict AST against the compiler's JSON Schema without type-checking or compiling. Returns a structured validation result indicating whether the AST is schema-valid, including any schema-violation errors with locations. Use this as a first pass before edict_check.
`edict_check`	no_return_description	Run the full semantic checker (name resolution, type checking, effect checking, contract verification) on an AST. Supports single module (ast) or multi-module (modules array) input. Returns structured checker results: lists of errors and warnings with severity, code, and location for any violations found.
`edict_patch`	no_return_description	Apply surgical patches to an Edict AST by nodeId, then run the full check pipeline — use this to fix errors without resubmitting the entire AST. Each patch specifies a nodeId, an operation (replace/delete/insert), and the relevant field/value. Returns the patched AST when returnAst is true, plus the errors/warnings produced by the check pipeline on the patched AST.
`edict_errors`	no_return_description	Return structured information about the error codes and categories produced by the Edict compiler, helping to interpret and resolve diagnostic messages.
`edict_export`	name_restates_behavior no_return_description	Compile an Edict AST and export it as a portable WASM skill package, running schema validation and generating the package manifest. Returns the SkillPackage JSON containing the embedded WASM, interface/entry-point metadata, and integrity checksum, ready for import via edict_import_skill.
`edict_import_skill`	name_restates_behavior no_return_description	Load a compiled Edict WASM skill package JSON (as produced by edict_export), verify its integrity checksum, and execute it in a sandboxed runtime with optional execution limits. Returns the execution result, including standard output, exit code, and any sandbox limit errors.
`edict_replay`	no_return_description	Re-execute a WASM module using a previously recorded replay token for deterministic reproduction of runtime behavior. All non-deterministic host responses (random values, timestamps, HTTP responses, file IO) are replayed from the token instead of calling real host functions. Returns the same structured execution result as edict_run (standard output, exit code, sandbox errors). Use this to reproduce exact failures or verify fixes against known execution traces.

Selection evidence

Confusable tool pairs.

7 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`edict_import_skill`	`edict_invoke_skill`	high	Both accept the same SkillPackage JSON and execute its WASM after checksum verification; only the 'import' vs 'execute' framing differs, so a task like 'run this packaged skill' or 'import and execute this skill' maps plausibly to either.
`edict_invoke_skill`	`edict_invoke`	medium	Both verbs are 'invoke'; invoke_skill runs a packaged WASM SkillPackage while invoke calls a deployed HTTP service, so 'invoke my skill' is ambiguous between executing the package and hitting the deployed endpoint.
`edict_export`	`edict_package`	medium	Both produce a portable skill package from an Edict module; export compiles from an AST while package requires precompiled WASM, so 'turn my module into a skill package' fits both and only the payload differs.
`edict_check`	`edict_compile`	low	Both take identical ast/modules inputs and are adjacent pipeline stages, so a task like 'verify and build my module' could trigger either, though check never returns WASM and compile assumes semantic validity.
`edict_compile`	`edict_deploy`	low	Deploy runs validate→check→compile and its wasm_binary target returns WASM+metadata just like compile, so 'compile my program to a wasm binary' can plausibly match either tool.
`edict_schema`	`edict_validate`	low	Both concern the AST JSON Schema; 'check my AST against the schema' could be read as fetching the schema to inspect or validating directly, though validate is the clear action.
`edict_errors`	`edict_explain`	low	With errors having an empty description and both containing 'error(s)', a task like 'explain the errors' or 'what does this error mean' is ambiguous between listing errors and returning enriched repair context for one error.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard