Public assessment

paladini/devutils-mcp-server (devutils-mcp-server)

devutils-mcp-server · v1.1.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 9%, unconfirmed writes 100%→100%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

30.0 / 30

30.0 out of 30

02Legibility

18.0 / 30

18.0 out of 30

03Economics

17.9 / 20

17.9 out of 20

04Discoverability

19.7 / 20

19.7 out of 20

Highest-impact fix

Estimated gain +1 point

Rewrite 31 underspecified tool descriptions

State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.

Description evidence

Defects and rewrites.

31 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`hash_md5`	name_restates_behavior no_return_description	Compute a one-way MD5 digest of the input string, useful as a fast checksum or cache key. Returns the 32-character lowercase hexadecimal hash.
`hash_sha1`	no_return_description	Compute a one-way SHA-1 digest of the input string, useful as a legacy 160-bit checksum. Returns the 40-character lowercase hexadecimal hash.
`hash_sha256`	no_return_description	Compute a one-way SHA-256 digest of the input string, useful for strong integrity checks. Returns the 64-character lowercase hexadecimal hash.
`hash_sha512`	no_return_description	Compute a one-way SHA-512 digest of the input string, useful for strong integrity verification. Returns the 128-character lowercase hexadecimal hash.
`hash_bcrypt`	name_restates_behavior no_return_description	Hash a password (or any string) with bcrypt for secure storage; higher rounds slow the hash against brute force. Returns the bcrypt hash string, which can later be checked with hash_bcrypt_verify.
`base64_encode`	name_restates_behavior no_return_description	Encode a string into Base64 so that binary or non-ASCII data can be embedded in text-based formats. Returns the Base64-encoded string.
`base64_decode`	name_restates_behavior no_return_description	Decode a Base64-encoded string back into its original content, the inverse of base64_encode. Returns the decoded plain string.
`url_encode`	name_restates_behavior no_return_description	Percent-encode a string so it is safe to place in a URL, query string, or form payload. Returns the string with special and non-ASCII characters replaced by percent-escapes.
`url_decode`	name_restates_behavior no_return_description	Decode a percent-encoded URL string back to its readable characters, e.g., to interpret query parameters. Returns the decoded string.
`html_encode`	name_restates_behavior no_return_description	Escape characters such as angle brackets, ampersands, and quotes into HTML entities to prevent markup breaking or injection. Returns the HTML-entity-encoded string.
`html_decode`	name_restates_behavior no_return_description	Convert HTML entities in a string back into their literal characters (for example ampersand-amp to ampersand) to restore the original text. Returns the decoded string.
`hex_encode`	no_return_description	Encode the input string into a hexadecimal representation so each byte is shown as two hex digits, useful for viewing raw data as text. Returns the hex string.
`hex_decode`	no_return_description	Decode a hexadecimal string back into its original text, the inverse of hex_encode. Returns the decoded plain string.
`generate_uuid`	name_restates_behavior no_return_description	Generate one or more RFC 4122 UUID v4 identifiers from a cryptographically secure random source, for use as unique IDs or keys. Returns the generated UUID string(s) as requested by count.
`generate_nanoid`	no_return_description	Generate a compact, URL-friendly NanoID of the requested length, useful for short opaque identifiers. Returns the generated ID string(s) as requested by count.
`generate_password`	name_restates_behavior no_return_description	Generate a random password of the requested length drawn from the enabled character sets (uppercase, lowercase, numbers, symbols), for use as a credential. Returns the generated password string(s) as requested by count.
`generate_random_hex`	no_return_description	Generate a random hexadecimal string from the specified number of random bytes, useful for tokens, salts, or test data. Returns a hex string of twice the requested byte length (2 times bytes characters).
`jwt_decode`	no_return_description	Decode a JWT and return its header and payload for inspection, useful for debugging; note that the signature is not verified, so the contents should not be trusted as authentic.
`jwt_validate`	no_return_description	Check a JWT structure for valid Base64URL segments, valid JSON, and non-expired status, without verifying the cryptographic signature. Returns a validation result indicating whether the token is valid and any errors found.
`json_format`	name_restates_behavior no_return_description	Pretty-print a JSON string with the given indentation to make it readable, or minify it by setting indent to 0. Returns the reformatted JSON string.
`json_validate`	name_restates_behavior no_return_description	Check that a JSON string is well-formed and report any parse errors with their approximate location. Returns the validation result along with any error details.
`json_path_query`	no_return_description	Extract the value at a dot-notation path, including array indexes, from a JSON document. Returns the value found at the path, or an indication when it is missing.
`timestamp_to_date`	name_restates_behavior no_return_description	Convert a Unix timestamp given in seconds or milliseconds into a human-readable ISO 8601 date string, optionally localized to an IANA timezone. Returns the formatted date string.
`date_to_timestamp`	name_restates_behavior no_return_description	Parse a date string in ISO 8601 or common date formats and convert it to a Unix timestamp, useful for math or comparisons. Returns the corresponding numeric Unix timestamp.
`number_base_convert`	name_restates_behavior no_return_description	Convert a numeric string between any bases 2-36, useful for switching between binary, octal, decimal, and hexadecimal representations. Returns the number expressed in the target base.
`color_convert`	name_restates_behavior no_return_description	Convert a color between HEX, RGB, and HSL formats, useful when matching color notations used by CSS, design tools, and APIs. Returns the color expressed in the other supported formats.
`byte_convert`	no_return_description	Convert a byte value from a source unit into the B, KB, MB, GB, TB, and PB units, using either the binary (1024) or SI (1000) standard. Returns the value expressed in each byte unit.
`lorem_ipsum`	name_restates_behavior no_return_description	Generate placeholder text for design mockups or layout previews, choosing the output type and quantity. Returns the requested number of Lorem Ipsum paragraphs, sentences, or words as a string.
`case_convert`	no_return_description	Convert a string to a target casing style such as camelCase, PascalCase, snake_case, kebab-case, CONSTANT_CASE, or Title Case. Returns the converted string.
`slugify`	no_return_description	Convert a string into a URL-friendly slug for paths and anchors, stripping special characters and lowercasing the result. Returns the slugified string.
`text_diff`	no_return_description	Compare two strings and produce a line-by-line diff, prefixing additions with '+', deletions with '-', and unchanged lines with ' '. Returns the diff output as a string.

Selection evidence

Confusable tool pairs.

23 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`timestamp_to_date`	`date_to_timestamp`	high	Both convert between timestamps and dates; a task like 'convert this date to a unix timestamp' or 'what date is 1736900000' could be reversed, and 'convert between timestamp and date' has no clear direction.
`html_encode`	`html_decode`	high	Directional pair; users frequently say 'encode this HTML' when they mean decoding entities back to text, and vice versa, so a vague 'convert this HTML/entity string' task is ambiguous.
`url_encode`	`url_decode`	medium	Both are 'url' plus an encode/decode verb in the same family; 'convert this URL string' or 'decode/encode this percent-encoded text' can lead to picking the wrong direction.
`hex_encode`	`hex_decode`	medium	Same hex string conversion with opposite direction; a task like 'convert this hex' or 'turn this string into hex/back to text' is ambiguous without an explicit encode versus decode verb.
`hash_bcrypt`	`hash_bcrypt_verify`	medium	A task like 'check this bcrypt hash' or 'verify/hash this password with bcrypt' is ambiguous between generating a hash and verifying a stored one; both share the bcrypt hash vocabulary.
`hash_sha1`	`hash_sha256`	medium	Both are SHA hashes of a string; a task like 'generate a SHA hash of this input' without naming the variant could pick either, producing different output.
`hash_sha1`	`hash_sha512`	medium	Same ambiguity: 'hash this with SHA' leaves the variant unspecified, and both tool descriptions are nearly identical apart from the algorithm name.
`hash_sha256`	`hash_sha512`	medium	Both generate a SHA-2 hash of the input; an underspecified task like 'get the SHA hash' or 'hash this string' could select either variant.
`hash_md5`	`hash_sha1`	medium	Both are generic 'hash this string' tools; a task that just says 'hash this value' or 'compute a one-way hash' gives no cue to pick MD5 over SHA-1.
`hash_md5`	`hash_sha256`	medium	Identical descriptions except algorithm name; 'generate a hash of this input string' is genuinely ambiguous between MD5 and SHA-256 when no algorithm is named.
`hash_md5`	`hash_sha512`	medium	Same generic 'hash a string' interface; without an explicit algorithm in the request, neither tool's description disambiguates the choice.
`hash_md5`	`hash_bcrypt`	medium	'Hash this password/string' could go either way since both accept a string to hash; the bcrypt 'password' hint is weak if the user only says 'hash this'.
`hash_sha1`	`hash_bcrypt`	medium	Both hash an input string; a task like 'hash this password securely' or just 'hash this' could select SHA-1 instead of bcrypt even though outputs differ.
`hash_sha256`	`hash_bcrypt`	medium	Both generate a hash of a given string; 'hash this input' or 'hash this password' is ambiguous because SHA-256 and bcrypt are both valid-looking hash tools.
`base64_decode`	`hex_decode`	medium	Both decode an encoded string back to plain text; a task like 'decode this string' without specifying base64 versus hex is genuinely ambiguous.
`base64_decode`	`url_decode`	medium	Both convert an encoded string back to plain text; 'decode this string/text' is ambiguous between a Base64 and a percent-encoded input without a format cue.
`base64_decode`	`jwt_decode`	medium	A JWT is base64url-encoded, so a task like 'decode this token/string' may lead the agent to generic base64_decode instead of the JWT-specific tool.
`base64_encode`	`hex_encode`	medium	Both encode a string into a text representation; 'encode this string' without naming base64 or hex could pick either tool.
`base64_encode`	`url_encode`	medium	Both encode a plain string; an underspecified 'encode this' or 'escape this text' task doesn't clearly distinguish base64 from percent-encoding.
`jwt_decode`	`jwt_validate`	medium	A task like 'check this JWT' is ambiguous between decoding/inspecting it and validating its structure and expiration; both share the 'jwt' token and token input.
`generate_uuid`	`generate_nanoid`	medium	Both just 'generate an ID/unique identifier'; a task like 'give me a random ID' does not reveal whether a UUID or a compact NanoID is wanted.
`generate_nanoid`	`generate_random_hex`	low	Tasks like 'generate a random token/ID string' could match either tool, though most requests are specific enough (nanoid for IDs, hex for hex output) to disambiguate.
`generate_nanoid`	`generate_password`	low	'Generate a random string/token' is weakly ambiguous, but password requests usually imply character-class options and memorable secrets, separating them from ID generation.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard