Public assessment

hifriendbot/cogmemai-mcp (cogmemai-mcp)

cogmemai-mcp · v3.21.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 96→96, token cost up 4%, unconfirmed writes 37%→43%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

11.0 / 30

11.0 out of 30

02Legibility

17.2 / 30

17.2 out of 30

03Economics

14.8 / 20

14.8 out of 20

04Discoverability

12.7 / 20

12.7 out of 20

Highest-impact fix

Estimated gain +19 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

31 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`save_memory`	no_return_description	Store a developer memory (fact, preference, decision, architecture detail). Memories persist across all Claude Code sessions and are available in future conversations. Returns the created memory record including its assigned memory_id, which you can later use to update, delete, or link the memory.
`save_rule`	no_return_description	Save a mandatory rule that will ALWAYS be followed in every session. Rules bypass scoring and decay and are injected into every conversation, so use them for absolute requirements such as 'NEVER do X' or 'ALWAYS do Y'. Returns the created rule with its memory_id, usable with list_rules and delete_rule.
`list_rules`	name_restates_behavior no_return_description	Retrieve the mandatory rules currently in effect for the current project and/or globally, optionally filtered by scope. Rules are always-on memories injected into every session, so this shows the standing requirements the assistant must obey. Returns a list of rules with their IDs, content, subject, scope, category, and tags.
`delete_rule`	no_return_description	Permanently remove a mandatory rule by its memory ID so it no longer applies in any future session. Find the ID first with list_rules. Returns confirmation that the rule was deleted.
`get_project_context`	no_return_description	Load the top memories for the current project plus relevant global memories to restore full context from previous sessions at session start. Pass context to rank memories toward your current task and context_type (debugging, planning, reviewing) to shift scoring weights. Returns formatted context text of the retrieved memories by default, or full JSON arrays when compact=false.
`list_memories`	name_restates_behavior no_return_description	Browse stored memories with optional filters by type, category, scope, importance, or tag, plus pagination and a configurable sort order (importance, updated, created, referenced, least_used). Returns a page of matching memory records, up to limit results starting at offset.
`delete_memory`	no_return_description	Move a memory to the trash by its ID so it is hidden from recall. It remains restorable via restore_memory for 30 days, then is purged permanently. Returns confirmation that the memory was trashed.
`restore_memory`	no_return_description	Bring a trashed (deleted) memory back into active use by its ID. Works within 30 days of deletion, before the trash is purged. Use list_trash to find restorable IDs. Returns the restored memory record.
`list_trash`	no_return_description	List the memories currently in the trash, newest first, showing how many days remain before each is permanently purged. Use restore_memory with one of the listed IDs to recover a deleted memory. Returns a paginated list of trashed memories with their IDs and purge countdown.
`update_memory`	name_restates_behavior no_return_description	Modify an existing memory's content, importance, scope, type, category, subject, or tags (tags replace existing ones). Useful for correcting a stored fact or reflecting a change in priority. Returns the updated memory record.
`bulk_delete`	no_return_description	Permanently delete multiple memories at once by their IDs, up to 100 per call. Unlike delete_memory, this bypasses the trash, so deletion is immediate and irreversible. Returns confirmation of how many memories were deleted.
`bulk_update`	no_return_description	Update up to 50 memories in a single call. Each item supplies a memory_id plus any fields to change (content, importance, scope, type, category, subject, tags). Returns the updated memory records.
`get_usage`	name_restates_behavior no_return_description	Check your CogmemAi account usage before deciding whether to save, export, or consolidate memories. Reports memory count, extractions used this month, tier info, and projects. Returns the usage statistics as a structured object.
`export_memories`	name_restates_behavior no_return_description	Download all memories as a single JSON payload, suitable for backing up or transferring them to another project. Returns the complete set of memories formatted as JSON.
`import_memories`	name_restates_behavior no_return_description	Bulk-load memories from a JSON string: pass an array of memory objects, each requiring at minimum a content field and optionally memory_type, category, subject, importance, or scope. Deduplication is applied automatically. Returns a summary of how many memories were imported or skipped.
`ingest_document`	no_return_description	Extract memories from a long document by splitting it into chunks and processing each one. Ideal for onboarding: feed in READMEs, architecture docs, or API specs to quickly build project context. Provide a document_type hint (readme, api_docs, architecture, changelog) to guide extraction. Returns the memories extracted from the document.
`save_session_summary`	no_return_description	Persist a summary of the current coding session — what was accomplished, key decisions, and next steps — stored as a session_summary memory for future reference. Returns confirmation of the saved summary.
`list_tags`	no_return_description	See every tag currently in use across your memories, with a count of memories per tag, to discover existing threads or groups and find related memories. Returns the tag list with counts.
`link_memories`	no_return_description	Connect two related memories with a named relationship (led_to, contradicts, extends, related) to build a knowledge graph — e.g., linking a bug fix to the architecture decision that caused it. Returns confirmation of the created link.
`get_memory_versions`	no_return_description	Review the edit history of a memory, showing all previous versions with timestamps and what changed, to understand how a decision or fact evolved over time. Returns an ordered list of versions with their content and change time.
`get_analytics`	no_return_description	Get a memory health dashboard with insights: most-recalled memories, never-recalled memories, stale memories, growth trends, and breakdowns by type and category, to identify cleanup opportunities. Optionally filter to a specific project or pass 'all' for cross-project analytics. Returns the analytics report.
`promote_memory`	no_return_description	Promote a project-scoped memory to global scope so it applies across all projects — useful when you discover a preference or pattern that should be universal, such as a coding preference. Returns confirmation of the scope change.
`consolidate_memories`	no_return_description	Reduce memory clutter by merging clusters of related memories (3+ sharing the same subject) into 1-2 richer facts; originals are archived, not deleted, with full version history. Use dry_run=true to preview without making changes. Returns the synthesized memories and the number of clusters processed.
`save_task`	no_return_description	Create a task that persists across sessions, tracked with status (pending, in_progress, done, blocked) and priority (high, medium, low), to maintain continuity on multi-session work. Returns the created task with its task_id.
`update_task`	no_return_description	Change a task's status, title, description, or priority — for example, marking it in_progress, done, or blocked as you work. Find the task_id with get_tasks. Returns the updated task.
`save_correction`	no_return_description	Record a correction pattern — what went wrong and the right approach — so it is surfaced automatically when similar situations arise in future sessions, helping avoid repeated mistakes. Returns confirmation of the saved correction.
`set_reminder`	no_return_description	Schedule a reminder that surfaces automatically at the start of your next session, for follow-ups, things to check, or deferred work. Reminders auto-archive after being shown. Returns confirmation of the scheduled reminder.
`get_file_changes`	no_return_description	Show what files changed since your last session by comparing the current git state to a snapshot saved at the end of the previous session, helping you understand what happened between sessions. Returns the list of changed files.
`feedback_memory`	no_return_description	Signal whether a recalled memory was useful or irrelevant: 'useful' boosts its future ranking (or skill confidence) and 'irrelevant' reduces its importance, improving recall quality over time. Returns confirmation that feedback was recorded.
`generate_skills`	no_return_description	Manually trigger skill generation from your corrections, preferences, and patterns, or preview candidates with dry_run=true. Skills are behavioral directives that auto-improve how the AI works with you; CogmemAi also generates them automatically when enough evidence accumulates. Returns the generated or previewed skills.
`extract_principles`	no_return_description	Analyze memory clusters to extract underlying factual principles or patterns about your project, such as 'this codebase tends to have X' — observations rather than behavioral instructions (those are skills). Use dry_run=true to preview candidates first. Returns the extracted principles.

Selection evidence

Confusable tool pairs.

23 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get_memory_links`	`get_memory_versions`	low	Both are single-memory read lookups with near-identical get_memory_* naming, so a vague task like 'show me everything about this memory' could route to either, but descriptions clearly split knowledge-graph connections from edit history, making real confusion unlikely.
`bulk_delete`	`bulk_update`	medium	Both take an array of memory IDs and operate on many memories in one call ('delete multiple memories at once' vs 'update multiple memories at once'); a task like 'apply a change to all these memories' or 'clean up these memories' without an explicit delete/update verb is ambiguous.
`delete_rule`	`delete_memory`	medium	Rules are stored as memories and both tools take a memory_id; the only differentiator is rule-vs-memory intent. A task like 'delete this rule I saved' can route to delete_memory (trash) instead of delete_rule, especially since delete_memory does not know about rule semantics.
`bulk_update`	`update_task`	medium	Tasks are a memory type and both tools are 'update' operations; a request to update several tasks at once could route to bulk_update (which edits memory fields) rather than update_task (which handles status/priority per task).
`delete_memory`	`bulk_delete`	medium	Both delete memories; delete_memory is single-item and reversible (goes to trash), bulk_delete is array-based and permanent. A task like 'delete these memories' or 'remove memory 5' is ambiguous about which tool and whether the action is destructive.
`delete_rule`	`bulk_delete`	medium	Deleting several rules at once has no dedicated bulk tool, so an agent may choose bulk_delete (which operates on memory IDs, and rules are memories) instead of repeatedly calling delete_rule; the intent 'remove multiple rules' is ambiguous between the two.
`export_memories`	`import_memories`	medium	Both are JSON-based memory transfer tools and export's description even mentions transferring to another project; a migration-style task like 'move my memories to the new project' or 'load my memories from a file' is ambiguous about direction.
`extract_memories`	`extract_principles`	medium	Both start with 'extract' and involve AI summarization; a task like 'extract the key takeaways/insights from my memories' could route to extract_memories (needs a conversation exchange) or extract_principles (scans memory clusters) since neither verb disambiguates the source.
`get_tasks`	`update_task`	medium	The colloquial phrase 'give me an update on my tasks' or 'update my task list' contains 'update' + 'task', which an agent could map to update_task (a mutation) when the user only wants to view current tasks, i.e., get_tasks.
`get_usage`	`get_analytics`	medium	Both return memory statistics: get_usage gives memory count/extractions/tier while get_analytics gives a health dashboard (recalled, stale, growth). A task like 'show me my memory stats' or 'how are my memories doing' is ambiguous between the two.
`import_memories`	`consolidate_memories`	low	Both 'merge' memories in some sense (import dedupes incoming JSON, consolidate synthesizes clusters), so a task like 'merge my backup memories in' vs 'merge my related memories' could be misrouted, but descriptions make source (file vs existing store) clear.
`link_memories`	`consolidate_memories`	medium	Both operate on related memories and share the 'related' token; a task like 'connect/organize these related memories' could map to link_memories (build graph edges) or consolidate_memories (merge clusters into richer facts), with opposite outcomes.
`list_memories`	`list_tags`	low	A task like 'show me how my memories are grouped' or 'what tags do I have' spans both (list_tags returns tags with counts, list_memories supports tag filtering), but the plain 'list memories' vs 'list tags' intent is usually distinct.
`list_memories`	`list_trash`	medium	Both list memory entries and a user saying 'show me all my memories' may expect trashed/deleted memories to appear, which list_memories does not do; the ambiguity is whether the request includes deleted items (list_trash) or not.
`list_rules`	`list_memories`	low	Rules are stored as memories (type 'rule'), so 'show me everything saved for this project' could route to list_memories or list_rules; however the tools clearly describe rules as special always-on memories, limiting genuine confusion.
`promote_memory`	`feedback_memory`	medium	Both 'boost' a memory's prominence: promote_memory raises scope to global, feedback_memory with 'useful' raises ranking. A task like 'make this memory more important/prioritize it' is ambiguous between the two.
`save_memory`	`save_correction`	medium	Both persist lessons for the future; a task like 'remember not to make this mistake again' could be saved as a correction (wrong_approach/right_approach) or as a regular memory (bug/pattern type), and the schemas differ materially.
`save_memory`	`save_rule`	high	Both take near-identical inputs (content, subject, scope, category, tags) and 'save this rule/remember this rule' is ambiguous: save_rule makes it mandatory and always injected, while save_memory just stores a normal memory; an agent must infer 'absolute requirement' intent to choose correctly.
`save_rule`	`save_correction`	medium	Both persist behavioral guidance; a task like 'save that I must always do X the right way' could be a mandatory rule (save_rule) or a correction pattern (save_correction with wrong/right approach), depending on how strongly the constraint is worded.
`save_task`	`update_task`	medium	A task like 'save this task as in_progress' is ambiguous between creating a new task via save_task (which has a status parameter) and updating an existing task via update_task; the user rarely clarifies whether the task exists already.
`update_memory`	`bulk_update`	low	Both update memories (single vs array); for a single memory either works functionally, and a task like 'update memory X' vs 'update a batch' is usually disambiguated by count, so wrong picks are rare and low-consequence.
`update_memory`	`promote_memory`	medium	update_memory accepts a 'scope' parameter that can set a memory to global, and promote_memory exists specifically to move project memories to global; a task like 'make this memory apply everywhere' is ambiguous between the two.
`get_analytics`	`get_stale_memories`	medium	get_analytics explicitly surfaces 'stale memories' in its dashboard while get_stale_memories exists specifically to find outdated memories; a task like 'show me my stale/forgotten memories' could route to either, with different output granularity.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard