Public assessment

Karakeep (@karakeep/mcp, Glama popular)

karakeep-mcp · v0.33.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 96→96, token cost up 11%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

20.4 / 30

20.4 out of 30

03Economics

17.6 / 20

17.6 out of 20

04Discoverability

10.4 / 20

10.4 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

28 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`get-asset`	name_restates_behavior no_return_description	Returns a time-limited signed URL that lets you download the file stored for an asset. Request it right before you need to fetch or stream the content, since the URL is only valid temporarily.
`search-bookmarks`	no_return_description	Search bookmarks using full-text, semantic, or hybrid matching, with optional qualifiers (is:fav, is:archived, #tag, list:<name>, url:<value>, date filters) to narrow results. Returns a page of matching bookmarks; use the returned nextCursor to fetch the next page.
`get-bookmark`	name_restates_behavior no_return_description	Fetches a single bookmark by its id and returns its full record, including url, title, note, summary, and timestamps, so you can inspect or edit it.
`get-bookmark-lists`	no_return_description	Returns every list that contains the given bookmark, showing all groupings the bookmark belongs to, which is useful for checking or changing its membership.
`create-bookmark`	name_restates_behavior no_return_description	Creates a new bookmark. Use type 'link' with a URL in content to save a web link, or type 'text' with the content to save a text note. Returns the created bookmark with its details.
`get-bookmark-content`	name_restates_behavior no_return_description	Retrieves the readable content of a bookmark rendered as markdown, which is the same source text used for text search and for creating highlights by character offset. Returns the markdown content for the given bookmark.
`delete-bookmark`	no_return_description	Permanently deletes a bookmark along with its highlights and assets. This cannot be undone. Returns a confirmation once the bookmark is removed.
`list-highlights`	name_restates_behavior no_return_description	Fetches highlights from all bookmarks, ordered newest first, and returns a page of them. Use limit and the returned cursor to page through more results.
`get-bookmark-highlights`	no_return_description	Returns every highlight attached to the given bookmark, including each highlight's text, color, note, and character offsets.
`get-highlight`	no_return_description	Fetches a single highlight by its id and returns its details (text, color, note, and character offsets), useful before updating or deleting it.
`create-highlight`	name_restates_behavior no_return_description	Adds a text highlight to a bookmark at the character range given by startOffset and endOffset, which are offsets into the bookmark's readable content as returned by get-bookmark-content. Returns the created highlight.
`update-highlight`	name_restates_behavior no_return_description	Changes an existing highlight's color or note; omit a field to leave it unchanged, and pass null for note to clear it. Returns the updated highlight.
`delete-highlight`	name_restates_behavior no_return_description	Removes a highlight from its bookmark. Returns a confirmation once it is deleted.
`get-lists`	no_return_description	Returns all lists available to the user, including each list's name, icon, and parent relationship, so you can browse lists and reference them by id.
`get-list`	no_return_description	Fetches a single list by its id and returns its details, including name, icon, description, parent list, smart-list query, and public status.
`update-list`	no_return_description	Modifies a list, changing only the fields you pass: name, icon, description, parent, smart-list query, or public status; pass null to clear the description or move to the root. Length caps and smart-list rules come from the shared list schema. Returns the updated list.
`delete-list`	no_return_description	Deletes a list by its id. Bookmarks inside it are not deleted, and child lists become root-level lists (their parentId is set to null); move or re-parent children first if you need a different tree change. Returns a confirmation of the deletion.
`add-bookmark-to-list`	name_restates_behavior no_return_description	Adds a bookmark to a list so the bookmark appears in that list's bookmarks and the list tracks it. Returns a confirmation.
`remove-bookmark-from-list`	name_restates_behavior no_return_description	Removes a bookmark from a list so the bookmark no longer appears in that list's bookmarks. Returns a confirmation.
`get-list-bookmarks`	no_return_description	Returns the bookmarks in a list, evaluated from its saved query for smart lists. Supports sorting by creation date, pagination via limit and cursor, and optionally each bookmark's full content when includeContent is set.
`create-list`	name_restates_behavior no_return_description	Creates a new list with the given name and emoji icon, optionally nested under a parent list. Returns the created list with its id and details.
`attach-tag-to-bookmark`	name_restates_behavior no_return_description	Applies one or more tags by name to a bookmark so they are associated with it. Returns a confirmation.
`detach-tag-from-bookmark`	name_restates_behavior no_return_description	Removes one or more tags by name from a bookmark so they are no longer associated with it. Returns a confirmation.
`get-tags`	no_return_description	Returns tags along with the number of bookmarks attached to each, supporting filtering by name substring and attachment method, sorting by name, usage, or relevance (relevance requires nameContains), and cursor pagination.
`get-tag`	no_return_description	Retrieve a single tag by its id. Returns the tag object, including its bookmark counts.
`update-tag`	no_return_description	Rename a tag by its id. The new name is normalized by the server. Returns the updated tag object with its normalized name.
`delete-tag`	no_return_description	Delete a tag by its id. Bookmarks that had this tag are not deleted; the tag is just removed from them. Returns the deleted tag object.
`get-tag-bookmarks`	no_return_description	List the bookmarks that have a given tag, with pagination. Returns a page of bookmarks and a cursor to fetch the next page. Optionally includes each bookmark's full content.

Selection evidence

Confusable tool pairs.

6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get-bookmark-lists`	`get-list-bookmarks`	medium	Mirror-image names ('lists of a bookmark' vs 'bookmarks of a list'); a task like 'show me the lists/bookmarks tied to this id' leaves ambiguous whether the supplied id is a bookmark or a list, and both tools take a single id.
`get-list`	`get-list-bookmarks`	medium	A task like 'show me list X' is ambiguous between retrieving the list's metadata and retrieving the bookmarks it contains; 'what's in this list' could map to either tool depending on how the agent reads 'in'.
`get-bookmark`	`get-bookmark-content`	medium	'Get the bookmark' vs 'get the bookmark's content' — a request like 'pull up bookmark X' or 'show me what's in this bookmark' could be satisfied by either the metadata-only get-bookmark or the full markdown from get-bookmark-content, and 'content' is a salient substring of both names.
`create-bookmark`	`create-highlight`	medium	Both involve 'creating a text' related to a bookmark: create-bookmark supports type=text while create-highlight takes a text param. A task like 'bookmark/highlight this text passage' is genuinely ambiguous between saving a text bookmark and creating a text highlight.
`get-tag`	`get-tag-bookmarks`	low	'Show me tag X' could mean the tag's details/counts (get-tag) or the bookmarks carrying that tag (get-tag-bookmarks); phrasing like 'what's in tag work' is ambiguous until the agent decides whether the user wants tag metadata or the tagged bookmarks.
`get-bookmark-content`	`get-bookmark-highlights`	low	A request like 'extract the text from this bookmark' could target the full markdown (get-bookmark-content) or the highlighted excerpts (get-bookmark-highlights); without an explicit 'highlights' keyword the two read similarly for an agent focused on text extraction.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard