Public assessment

cameronrye/activitypub-mcp (activitypub-mcp)

activitypub-mcp · v3.2.1 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 6%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

25.6 / 30

25.6 out of 30

03Economics

15.2 / 20

15.2 out of 20

04Discoverability

17.5 / 20

17.5 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`discover-instances`	name_restates_behavior	Find fediverse servers matching your criteria by querying the live instances.social directory. Filter by software type, language, user counts, and whether registrations are open; control result ordering with sortBy/sortOrder and cap the number of results with limit. Returns a list of matching instances (domain, software, user count, and description) plus a hasMore flag indicating whether more results exist.
`get-trending-hashtags`	name_restates_behavior	Retrieve the hashtags that are currently popular on a given Mastodon-compatible fediverse instance (the instance must expose a public trends API). Use this to identify hot topics on a server. Pass the instance domain and optionally limit the number returned. Returns hashtag entries with name, canonical URL, and usage counts (uses and accounts).
`get-trending-posts`	name_restates_behavior	Retrieve posts that are currently trending on a given Mastodon-compatible fediverse instance (the instance must expose a public trends API). Pass the instance domain and optionally limit the number returned. Returns trending posts with content, author, timestamps, and interaction counts (replies, reblogs, favourites), plus a nextCursor and hasMore flag for pagination.
`get-notifications`	name_restates_behavior params_unexplained	Return the notifications received by an authenticated account, covering mentions, follows, boosts (reblogs), favourites, follow requests, polls, and status updates. Pass accountId to select which configured account's notifications to fetch, use types to filter to specific notification categories, and limit to cap the number returned. Returns a list of notifications, each with its type, the originating account, the related status (if any), and a creation timestamp.
`get-home-timeline`	params_unexplained	Fetch the personalized home timeline of an authenticated account, showing posts from the accounts it follows. Pass accountId to choose which configured account's timeline to return, and use limit, maxId, and sinceId to control how many posts are returned and paginate by ID. Returns posts with content, author, timestamps, and interaction counts, plus a nextCursor and hasMore flag for pagination.
`get-bookmarks`	params_unexplained	Return the posts that an authenticated account has bookmarked. Pass accountId to choose which configured account's bookmarks to fetch and limit to cap the number returned. Returns posts with content, author, timestamps, and interaction counts, plus a nextCursor and hasMore flag for pagination.
`get-favourites`	params_unexplained	Return the posts that an authenticated account has favourited (liked). Pass accountId to choose which configured account's favourites to fetch and limit to cap the number returned. Returns posts with content, author, timestamps, and interaction counts, plus a nextCursor and hasMore flag for pagination.
`get-scheduled-posts`	params_unexplained	Return the posts that an authenticated account has scheduled for future publication and has not yet published. Pass accountId to choose which configured account's scheduled posts to fetch and limit to cap the number returned. Returns scheduled posts with their id, scheduled publication time, text, and visibility.

Selection evidence

Confusable tool pairs.

8 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get-trending-hashtags`	`get-trending-posts`	high	Both fetch 'currently trending' items from the same Mastodon instance with identical input schemas (domain + limit); a task like 'what's trending on mastodon.social?' names neither item type, so the agent must guess hashtags vs posts.
`get-bookmarks`	`get-favourites`	high	Both retrieve the user's own saved posts with nearly identical schemas (limit, accountId); 'get my saved posts' or 'show my bookmarks/favourites' maps equally well to either since bookmarks and favourites are conceptually overlapping collections.
`get-public-timeline`	`get-home-timeline`	medium	Both return a timeline of posts; an unqualified task like 'show me the timeline' or 'get my feed' could route to the instance's public timeline instead of the personalized home timeline when the task doesn't specify 'public' vs 'mine'.
`fetch-timeline`	`get-home-timeline`	medium	A task like 'get the timeline posts for my account' is ambiguous: fetch-timeline returns any actor's outbox given a handle, while get-home-timeline returns 'your' feed keyed by accountId; both descriptions match account + timeline + posts.
`fetch-timeline`	`get-public-timeline`	medium	A generic 'fetch the timeline / recent posts' task matches both: fetch-timeline grabs a single actor's outbox while get-public-timeline grabs an instance's federated/local feed, so without a domain-vs-actor target the wrong timeline tool can be chosen.
`discover-instances`	`get-instance-info`	medium	Tasks phrased as 'find/show info about fediverse instances' are ambiguous: discover-instances returns a filtered live list of instances while get-instance-info returns details for one specific domain; wording like 'get instance info' doesn't disambiguate discovery from inspection.
`get-notifications`	`get-favourites`	low	get-notifications lists 'favourites' as a notification type, so 'check my favourites' could be misrouted to notifications filtered by type before the agent notices the dedicated get-favourites tool; plausible but the exact verb match in the dedicated tool usually resolves it.
`discover-actor`	`discover-instances`	low	Both are 'discover the fediverse' tools sharing the discover verb, so a vague task like 'discover something on the fediverse' gives no profile-vs-server signal; normal tasks name either a handle or software filters, so real confusion is unlikely.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard