Public assessment

cameronrye/activitypub-mcp (activitypub-mcp)

activitypub-mcp · v3.2.1 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 2%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

26.3 / 30

26.3 out of 30

03Economics

14.8 / 20

14.8 out of 20

04Discoverability

11.9 / 20

11.9 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`discover-instances`	name_restates_behavior	Query the instances.social directory for fediverse servers matching filters like software, language, user-count range, and open registration, returning a sortable, paginated list of matching instance domains with basic stats.
`get-trending-hashtags`	name_restates_behavior	Query a Mastodon-compatible instance's trends API for its current top hashtags, returning each tag's name, URL, and recent usage/account counts.
`get-trending-posts`	name_restates_behavior	Query a Mastodon-compatible instance's trends API for its current top posts, returning post content, author, engagement counts, and a pagination cursor.
`get-home-timeline`	params_unexplained	Fetch the personalized home timeline (posts from accounts you follow) for a configured authenticated account, returning posts with content, author, timestamps, and engagement counts. Supports paging via maxId/sinceId and limit; accountId selects which configured account to use (defaults to the active account from list-accounts).
`get-notifications`	name_restates_behavior params_unexplained	Fetch notifications (mentions, follows, boosts, favourites, polls, updates) for a configured authenticated account, returning each notification's type, source account, related post, and timestamp. Supports filtering by notification type and limiting result count; accountId selects which configured account to use (defaults to the active account from list-accounts).
`get-bookmarks`	params_unexplained	Fetch the bookmarked posts saved by a configured authenticated account, returning post content, author, and engagement counts with a pagination cursor. accountId selects which configured account to use (defaults to the active account from list-accounts).
`get-favourites`	params_unexplained	Fetch the posts favourited by a configured authenticated account, returning post content, author, and engagement counts with a pagination cursor. accountId selects which configured account to use (defaults to the active account from list-accounts).
`get-scheduled-posts`	params_unexplained	List the not-yet-published scheduled posts for a configured authenticated account, returning each post's ID, scheduled time, text, and visibility. accountId selects which configured account to use (defaults to the active account from list-accounts).

Selection evidence

Confusable tool pairs.

6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get-bookmarks`	`get-favourites`	high	Both return 'posts you saved' with identical schemas (limit, accountId); a user asking to 'show my saved posts' or 'liked posts' could easily be routed to the wrong one since bookmark vs favourite is a Mastodon-specific distinction not obvious from casual phrasing.
`get-trending-hashtags`	`get-trending-posts`	medium	Identical schema (domain, limit) and near-identical descriptions differing only in 'hashtags' vs 'posts'; a vague request like 'what's trending on mastodon.social' without specifying hashtags or posts could be mapped to either.
`get-public-timeline`	`get-home-timeline`	medium	Both are 'timeline' fetchers; a request like 'show me the latest posts' is ambiguous between the instance-wide public timeline (needs a domain) and the personalized home timeline (posts from accounts followed), especially if the agent doesn't carefully track which one requires no domain vs requires one.
`fetch-timeline`	`get-home-timeline`	medium	Both fetch a 'timeline' of posts; 'get my timeline' could be misrouted to fetch-timeline (which fetches any actor's outbox given a handle) instead of get-home-timeline (the personalized feed of followed accounts), particularly if the agent tries to pass the user's own handle to fetch-timeline.
`fetch-timeline`	`get-public-timeline`	low	Both take pagination-like params and relate to 'timeline' content, but fetch-timeline is scoped to a specific actor's posts while get-public-timeline is instance-wide; confusion is plausible only if the task vaguely says 'get posts from mastodon.social' without clarifying a specific user vs the instance feed.
`discover-instances`	`get-instance-info`	low	Both relate to fediverse instances, but discover-instances searches/filters across many instances while get-instance-info fetches details for one named domain; a task like 'tell me about good mastodon instances' could blur the two if the domain isn't already specified.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard