Public assessment

paladini/devutils-mcp-server (devutils-mcp-server)

devutils-mcp-server · v1.1.0 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 5%, unconfirmed writes 100%→100%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

30.0 / 30

30.0 out of 30

02Legibility

20.0 / 30

20.0 out of 30

03Economics

17.7 / 20

17.7 out of 20

04Discoverability

13.9 / 20

13.9 out of 20

Highest-impact fix

Estimated gain +1 point

Rewrite 31 underspecified tool descriptions

State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.

Description evidence

Defects and rewrites.

31 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`hash_md5`	name_restates_behavior no_return_description	Computes the MD5 digest of the given input string and returns it as a hexadecimal string. Note MD5 is not cryptographically secure and should not be used for security-sensitive purposes.
`hash_sha1`	no_return_description	Generate a SHA-1 hash of the given input string and return it as a hexadecimal string.
`hash_sha256`	no_return_description	Generate a SHA-256 hash of the given input string and return it as a hexadecimal string.
`hash_sha512`	no_return_description	Generate a SHA-512 hash of the given input string and return it as a hexadecimal string.
`hash_bcrypt`	name_restates_behavior no_return_description	Hashes the given input string with bcrypt using the specified number of salt rounds (default 10), returning the resulting bcrypt hash string. Suitable for securely storing passwords.
`base64_encode`	name_restates_behavior no_return_description	Encodes the given string as Base64 and returns the encoded result.
`base64_decode`	name_restates_behavior no_return_description	Decodes the given Base64 string back into plain text and returns the decoded result.
`url_encode`	name_restates_behavior no_return_description	Percent-encodes the given string for safe use in a URL and returns the encoded result.
`url_decode`	name_restates_behavior no_return_description	Decodes a percent-encoded (URL-encoded) string and returns the decoded plain text.
`html_encode`	name_restates_behavior no_return_description	Escapes special characters in the given string into HTML entities and returns the encoded result, useful for safely embedding text in HTML.
`html_decode`	name_restates_behavior no_return_description	Converts HTML entities in the given string back into their original characters and returns the decoded result.
`hex_encode`	no_return_description	Encode a string to its hexadecimal representation and return the resulting hex string.
`hex_decode`	no_return_description	Decode a hexadecimal string back to plain text and return the decoded string.
`generate_uuid`	name_restates_behavior no_return_description	Generates one or more cryptographically secure random UUID v4 identifiers (count configurable, default 1) and returns them.
`generate_nanoid`	no_return_description	Generate a NanoID — a compact, URL-friendly unique ID — with configurable length and count, and return the generated ID(s).
`generate_password`	name_restates_behavior no_return_description	Generates one or more secure random passwords using configurable length and character set options (uppercase, lowercase, numbers, symbols) and returns them.
`generate_random_hex`	no_return_description	Generate a cryptographically random hexadecimal string from the specified number of random bytes and return the resulting hex string (twice the byte length).
`jwt_decode`	no_return_description	Decode a JWT (JSON Web Token) and return its header and payload contents without verifying the signature. Useful for debugging and inspecting tokens.
`jwt_validate`	no_return_description	Validate the structure of a JWT — checking format, Base64URL encoding, JSON validity, and expiration status — and return the validation result. Does NOT verify the cryptographic signature.
`json_format`	name_restates_behavior no_return_description	Formats a JSON string with configurable indentation (or minifies it when indent is 0) and returns the formatted JSON string.
`json_validate`	name_restates_behavior no_return_description	Validates whether a string is well-formed JSON and returns the validation result, including any parsing errors with their approximate location.
`json_path_query`	no_return_description	Extract a value from a JSON object using a dot-notation path (e.g., 'user.address.city' or 'items[0].name') and return the extracted value.
`timestamp_to_date`	name_restates_behavior no_return_description	Converts a Unix timestamp (seconds or milliseconds) into a human-readable ISO 8601 date string in the given timezone (default UTC), and returns that date string.
`date_to_timestamp`	name_restates_behavior no_return_description	Parses a date string (ISO 8601 or common formats) and returns the corresponding Unix timestamp.
`number_base_convert`	name_restates_behavior no_return_description	Converts a number string from one base to another (binary, octal, decimal, hexadecimal, or any base 2-36) and returns the converted value as a string.
`color_convert`	name_restates_behavior no_return_description	Parses a color value in HEX, RGB, or HSL format and returns its equivalent representation in the other supported formats.
`byte_convert`	no_return_description	Convert between byte units (B, KB, MB, GB, TB, PB), using either binary (1024) or SI (1000) standard, and return the converted value in the requested unit.
`lorem_ipsum`	name_restates_behavior no_return_description	Generates Lorem Ipsum placeholder text as paragraphs, sentences, or words (count configurable) and returns the generated text.
`case_convert`	no_return_description	Convert a string between different casing styles (camelCase, PascalCase, snake_case, kebab-case, CONSTANT_CASE, Title Case, and more) and return the converted string.
`slugify`	no_return_description	Convert a string to a URL-friendly slug by removing special characters, replacing spaces with hyphens, and lowercasing everything, then return the resulting slug.
`text_diff`	no_return_description	Compare two strings and return a simple line-by-line diff, where lines prefixed with '+' are additions, '-' are deletions, and ' ' are unchanged.

Selection evidence

Confusable tool pairs.

15 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`hash_sha1`	`hash_sha256`	medium	A vague request like 'hash this string' with no algorithm specified could plausibly map to either SHA-1 or SHA-256 since both have near-identical descriptions differing only in the algorithm name.
`hash_sha1`	`hash_sha512`	medium	Same as sha1/sha256: descriptions are otherwise identical, so an underspecified 'sha hash' request could pick either.
`hash_sha256`	`hash_sha512`	medium	Both are generic SHA hashing tools with identical descriptions apart from bit length; a user saying 'give me a SHA hash' without specifying length creates real ambiguity.
`hash_md5`	`hash_sha1`	low	Both are simple string hash generators, but MD5 and SHA-1 are commonly named explicitly by users, and MD5's distinct name reduces confusion versus the SHA family mixups.
`hash_bcrypt`	`hash_bcrypt_verify`	low	Both mention bcrypt and hash, but one requires only 'input' and generates a hash while the other requires 'input'+'hash' and checks a match — a task like 'check if this password matches' vs 'hash this password' is fairly distinguishable, though a careless agent could still mix them up.
`timestamp_to_date`	`date_to_timestamp`	medium	Both convert between the same two representations in opposite directions; a task phrased ambiguously like 'convert this date/timestamp' without clear directionality could cause the agent to pick the wrong one.
`base64_decode`	`hex_decode`	low	Both decode an encoded string back to plain text, but the encoding type (base64 vs hex) is usually evident from the input format, limiting real confusion.
`html_encode`	`html_decode`	low	Directionally opposite (encode vs decode) with clear verb distinction, but a careless 'convert HTML entities' request without clear direction could still cause mix-ups.
`url_encode`	`url_decode`	low	Clear encode/decode distinction in names and descriptions makes confusion unlikely except in vague 'process this URL string' requests.
`hex_encode`	`hex_decode`	low	Encode/decode direction is explicit in both name and description, so confusion is unlikely except for extremely vague 'convert to/from hex' phrasing.
`jwt_decode`	`jwt_validate`	medium	A request like 'check this JWT token' is ambiguous between decoding its contents for inspection and validating its structure/expiration, since both operate on tokens without verifying signatures.
`json_format`	`json_validate`	low	Both take a JSON string, but 'format/pretty-print' vs 'validate for errors' are distinct enough intents that confusion is unlikely except for a vague 'check this JSON' request.
`generate_uuid`	`generate_nanoid`	low	Both generate unique IDs, but UUID and NanoID are distinct enough named concepts that an explicit request would rarely be ambiguous; only a generic 'generate a unique ID' request creates mild ambiguity.
`generate_nanoid`	`generate_random_hex`	low	Both generate random identifier-like strings; a vague 'generate a random ID/string' request could plausibly go to either, though NanoID and hex have different typical use-cases.
`number_base_convert`	`byte_convert`	low	Both are 'convert between X' tools, but byte units (KB/MB) vs number bases (binary/hex) are conceptually distinct enough that only a very vague 'convert this number' request would cause confusion.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard