Public leaderboard

Public assessment

teampolarity/cosmos-mcp (@polarity-lab/cosmos-mcp)

polarity-lab-cosmos-mcp · v0.9.36 · scanned

What changed in the harness

Selection accuracy 95→97, token cost up 16%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30
02Legibility

24.2 / 30

24.2 out of 30
03Economics

19.6 / 20

19.6 out of 20
04Discoverability

11.8 / 20

11.8 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool Defect types Suggested rewrite
polarity_get_graph
no_return_description
Read the user's graph view. `entity` selects which projection: 'user' (the user's self-graph), 'cosmos' (the cosmos entity's view of them), or 'polarity' (the dyadic synchronization between the two). Use 'user' for general questions about what the user thinks, does, or knows. Use 'polarity' when comparing the user's self-image against the system's observation. Returns the selected graph projection: the nodes and edges of the user's knowledge graph for the requested `entity`.
polarity_observe
params_unexplained no_return_description
Single-fact write into the user's personal cosmos graph. Prefer polarity_capture_turn for whole exchanges; use polarity_observe only when one specific durable observation needs to land without the surrounding turn. Examples that warrant an inline call: the user states a hard rule, names a concrete preference, or corrects an assumption you had wrong. Avoid logging ephemeral chat or your own reasoning. `text` is the observation to record (the rule, preference, or correction stated by the user). `source` should identify your client (e.g. 'claude-code', 'cursor', 'claude-desktop'). `kind` defaults to 'observation'; use 'event' for things that happened, 'preference' for stated likes/dislikes. `tags` is an optional list of short labels to attach for later retrieval. `confidence` is an optional number from 0 to 1 expressing how certain you are the observation is accurate. Returns confirmation that the observation was recorded, including the created node id.
polarity_record_event
params_unexplained no_return_description
Record a structured event in the user's graph. Convenience wrapper over polarity_observe with kind='event'. Use for things that happened at a point in time: a meeting, a shipped release, a flight, an incident. `text` is a description of the event. `source` should identify your client (e.g. 'claude-code', 'cursor', 'claude-desktop'). `tags` is an optional list of short labels to attach to the event. `confidence` is an optional number from 0 to 1 expressing how certain you are the event occurred as described. Returns confirmation that the event was recorded, including the created node id.
polarity_record_preference
params_unexplained no_return_description
Record a stated preference in the user's graph. Convenience wrapper over polarity_observe with kind='preference'. Use when the user expresses a like, dislike, opinion, or working-style rule that should persist across sessions. `text` is the stated preference to record. `source` should identify your client (e.g. 'claude-code', 'cursor', 'claude-desktop'). `tags` is an optional list of short labels to attach to the preference. `confidence` is an optional number from 0 to 1 expressing how certain you are the preference was stated as described. Returns confirmation that the preference was recorded, including the created node id.
polarity_capture_turn
params_unexplained
This is how the user's cosmos graph learns from this conversation. CALL THIS AT THE END OF EVERY SUBSTANTIVE USER/ASSISTANT EXCHANGE, without being asked. The user installed cosmos-mcp precisely so their agents would persist what they notice. If you finish a turn and do not call this, that turn is lost to every other agent the user works with tomorrow. Cosmos runs its own extractor on the text and decides what is durable - your job is to hand over the raw exchange, not to pre-filter. Pass the user's message in `user_text` and your own reply in `assistant_text`. `max_observations` optionally caps how many durable items the extractor may return for this turn (1-20). Skip only for trivial single-word exchanges ('hi', 'thanks') and explicit meta-discussion of the tool itself. `source` should identify your client (e.g. 'claude-code', 'claude-desktop', 'cursor', 'codex', 'zed'). Returns the node ids cosmos created. Cheap to call; the extractor returns zero items if nothing was worth holding, and that is a fine outcome.
polarity_dump
params_unexplained no_return_description
Write a short message tied to a location waypoint into the user's graph. PolarityGPS-style. Use only when the user is explicitly recording a place-anchored thought. `waypoint_id` is the stable identifier of the waypoint, `name` its human-readable label, and `message` the place-anchored thought to store. `lat` and `lon` are optional coordinates for the waypoint. Returns confirmation that the message was written to the waypoint, including the waypoint's node id.
polarity_checkin
params_unexplained no_return_description
Record that the user checked in at a waypoint. Triggers co-presence detection against other users' recent check-ins. `waypoint_id` is the stable identifier of the waypoint and `name` its human-readable label; `lat` and `lon` are optional coordinates; `occurred_at` is the check-in time as an ISO 8601 date-time string (defaults to now if omitted). Returns the check-in record created in the graph.
polarity_declare
params_unexplained no_return_description
Declare future presence at a waypoint. `waypoint_id` is the stable identifier of the waypoint and `name` its human-readable label; `lat` and `lon` are optional coordinates. `starts_at` and `ends_at` are the declared presence window as ISO 8601 date-time strings, and `chip` is the time-window enum: next_30, next_hour, tonight, tomorrow_night. Returns the declaration record created in the graph.

Selection evidence

Confusable tool pairs.

6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A Tool B Confidence Why they collide
polarity_record_event polarity_record_preference medium Both are kind-wrappers over polarity_observe with identical schemas; a user statement bundling an event and a stance (e.g. 'I really liked the release we shipped') genuinely fits both, forcing the agent to decide which wrapper's kind captures the intent.
polarity_observe polarity_capture_turn medium Both persist user knowledge; observe handles a single durable fact while capture_turn ingests the raw exchange. A task like 'save that I prefer X from this conversation' could plausibly go either way, and the observe docs explicitly defer to capture_turn, blurring the boundary.
polarity_record_event polarity_capture_turn medium record_event's own example (a shipped release) overlaps capture_turn's mandate to persist every substantive exchange; a task like 'log that we shipped today' maps to the dedicated event wrapper by example but to capture_turn by the always-call rule.
polarity_checkin polarity_declare medium Both are waypoint writes with identical location fields; a task phrased 'I'll check in at the gym tonight' describes future presence that should map to declare, but the wording 'check in' pulls the agent toward checkin.
polarity_dump polarity_checkin low Both write to a waypoint with the same location schema; 'leave a mark/note at the cafe' vs 'record I was there' can be phrased ambiguously, though presence vs place-anchored message usually disambiguates.
polarity_whoami polarity_get_graph low An identity question like 'who am I according to cosmos' could match whoami (bound account info) or get_graph with entity=cosmos (the cosmos entity's view of the user); descriptions separate these but surface phrasing overlaps.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard