Public leaderboard

Public assessment

Notion MCP (official, listTools works unauthenticated)

notion-mcp · v1.0.0 · scanned

What changed in the harness

Selection accuracy 91→93, token cost up 2%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30
02Legibility

17.8 / 30

17.8 out of 30
03Economics

7.8 / 20

7.8 out of 20
04Discoverability

11.0 / 20

11.0 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

24 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool Defect types Suggested rewrite
API-get-user
params_unexplained no_return_description
Retrieves a Notion user by their unique user_id (a UUID) and returns the user object containing the user's name, avatar, type, and bot account details.
API-get-users
no_return_description
Lists all users in the workspace and returns a paginated list of user objects. Provide start_cursor to get results after a given page and page_size (maximum 100) to control how many users are returned per page.
API-get-self
no_return_description
Retrieves the bot user associated with the current integration's token and returns that user object, including its name, avatar, and bot details.
API-post-search
no_return_description
Searches pages and data sources by comparing the query text against their titles and returns a paginated list of matching objects. Use filter to restrict results to only pages or only data sources, sort to order results by last edited time, and start_cursor/page_size to control pagination.
API-get-block-children
no_return_description
Retrieves the immediate child blocks of the block or page identified by block_id and returns a paginated list of block objects. Use start_cursor and page_size (maximum 100) to control pagination.
API-patch-block-children
no_return_description
Appends an array of child block objects to the container block or page identified by block_id and returns the list of the newly appended block objects. Optionally pass after with the ID of an existing block to insert the new blocks immediately after it.
API-retrieve-a-block
no_return_description
Retrieves a single Notion block by its block_id and returns the block object containing the block's type, content, parent, and metadata.
API-update-a-block
no_return_description
Updates an existing block by its block_id, modifying text content for supported block types, updating the checked state of to_do blocks, or archiving/un-archiving the block via archived, and returns the updated block object.
API-delete-a-block
no_return_description
Deletes (archives) the Notion block identified by block_id and returns the archived block object. The deletion can be undone by un-archiving the block.
API-retrieve-a-page
no_return_description
Retrieves a Notion page by its page_id and returns the page object including its metadata and property values. Optionally pass filter_properties as a list of page property value IDs to limit the returned properties.
API-patch-page
params_unexplained no_return_description
Updates an existing Notion page's properties (keyed by property name or ID), icon (emoji or external file object), cover (external file object), or archived/in_trash status and returns the updated page object. Set in_trash or archived to true to delete the page, or to false to restore it.
API-post-page
params_unexplained no_return_description
Creates a new Notion page as a child of a parent page, data source, or the workspace, and returns the created page object including its URL and property values. Provide parent (a page_id, database_id, or workspace reference), properties (initial property values keyed by name or ID), and optionally children (block content to render on the page), plus an icon and cover.
API-retrieve-a-page-property
name_restates_behavior no_return_description
Retrieves the value of a specific property on a page, identified by page_id and property_id, and returns the property item object(s). Use page_size and start_cursor to paginate multi-item properties such as relations or rollups.
API-retrieve-a-comment
no_return_description
Retrieves the comments made on the block or page identified by block_id and returns a paginated list of comment objects. Use start_cursor and page_size (maximum 100) to control pagination.
API-create-a-comment
params_unexplained no_return_description
Creates a comment on a page, identifying the page with parent (containing the page_id) and supplying the comment text as rich_text, and returns the created comment object.
API-query-data-source
name_restates_behavior params_unexplained no_return_description
Queries a Notion data source (database) for its pages and returns a paginated list of matching page objects. Use filter to narrow the results, filter_properties to limit which property values are returned, sorts to order the results, start_cursor and page_size to control pagination, and archived or in_trash to include deleted pages.
API-retrieve-a-data-source
name_restates_behavior no_return_description
Retrieves a Notion data source (database) by its data_source_id and returns the data source object including its title, description, and property schema.
API-update-a-data-source
name_restates_behavior params_unexplained no_return_description
Updates a Notion data source (database) identified by data_source_id, providing a new title and description as rich text and/or property schema updates via properties, and returns the updated data source object.
API-create-a-data-source
name_restates_behavior params_unexplained no_return_description
Creates a new data source (database) as a child of the page identified by parent.page_id, with a property schema defined by properties, and returns the created data source object. Optionally provide a title as rich text.
API-list-data-source-templates
name_restates_behavior params_unexplained no_return_description
Lists the templates available in the data source (database) identified by data_source_id and returns a paginated list of template objects. Use start_cursor and page_size to control pagination.
API-retrieve-a-database
no_return_description
Retrieves a Notion database by its database_id and returns the database object including its title, description, property schema, and parent.
API-move-page
params_unexplained no_return_description
Moves the page identified by page_id to a new parent, which can be another page (type page_id with page_id), a data source (type database_id with database_id), or the workspace (type workspace), and returns the moved page object.
API-retrieve-page-markdown
no_return_description
Retrieves the Notion page (or block) identified by page_id as enhanced Markdown and returns the rendered Markdown content. Set include_transcript to true to include full meeting note transcripts; when false, a placeholder with the meeting note URL is returned instead.
API-update-page-markdown
no_return_description
Updates the content of the Notion page (or block) identified by page_id using Markdown and returns the page's resulting Markdown content. Choose type replace_content to overwrite the entire page or update_content for targeted find-and-replace edits, and provide the corresponding option object for the selected type; insert_content and replace_content_range are deprecated alternatives.

Selection evidence

Confusable tool pairs.

14 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A Tool B Confidence Why they collide
API-get-user API-get-users medium Singular/plural trap: a task like 'get the user accounts' or 'fetch the user directory' can route to API-get-user because 'user' is embedded in both names, while API-get-users says 'List all users'; without a clear ID, 'get the user(s)' is ambiguous.
API-get-user API-get-self medium A task like 'retrieve the user' or 'get my user info' is ambiguous between a specific user (needs user_id) and the token's own bot user; API-get-self takes no arguments, so a user lookup without an ID easily picks the wrong member.
API-get-users API-get-self low 'List all users' vs 'your token's bot user' are usually distinguishable, but a vague task 'get the user info' with no ID leaves both as candidates, since one returns a single user and the other a list.
API-retrieve-a-page API-retrieve-a-page-property medium Both are 'retrieve' + 'page'; a task like 'get the page's property values' could pick API-retrieve-a-page (which returns all properties) or API-retrieve-a-page-property (a single property item), especially when the user doesn't name a specific property ID.
API-retrieve-a-page API-retrieve-page-markdown medium 'Fetch the page' vs 'fetch the page as Markdown' overlap because both return page content; a task like 'read the page content' or 'export the page' is ambiguous when the desired format (JSON object vs Markdown) isn't stated.
API-get-block-children API-patch-block-children low Both names share 'block-children'; a generic task like 'update/modify the page's children' is ambiguous between reading children (get) and appending them (patch), though explicit 'append'/'list' tasks are usually clear.
API-query-data-source API-retrieve-a-data-source medium Classic confusion: 'query' returns rows, 'retrieve' returns schema/metadata. A task like 'fetch the data from the database' can route to API-retrieve-a-data-source since 'get/retrieve' implies fetching, while the records actually require querying.
API-retrieve-a-block API-retrieve-a-page low Pages are blocks and both endpoints accept an ID, so a task like 'get that block' pointing at a page (or vice versa) can pick the wrong resource type; descriptions don't clarify that page IDs are valid block IDs.
API-update-a-block API-delete-a-block medium API-update-a-block's archived field explicitly 'archive (delete)'s a block, so a task like 'archive/remove this block' fits both tools; an agent may pick update-a-block expecting the archived flag rather than delete-a-block.
API-patch-block-children API-patch-page medium Both are 'patch' operations on a page-ish target; a generic task like 'update the page by adding a section' could route to API-patch-page (properties/icon/cover) instead of API-patch-block-children (appending content blocks).
API-patch-page API-update-page-markdown medium Both update a page; 'edit the page' is ambiguous between changing properties (patch) and changing content as Markdown (update-page-markdown). A task providing Markdown text may still pick API-patch-page because of the page focus.
API-update-a-block API-update-page-markdown medium API-update-page-markdown accepts a 'page (or block)' ID, so 'update this block's content as Markdown' fits both tools; API-update-a-block only supports text/checked fields, making wrong selection likely when content editing is requested.
API-patch-block-children API-update-a-block medium 'Update the block with new text' is ambiguous between editing the existing block (update-a-block) and appending new child content (patch-block-children), e.g., adding vs. rewriting a bulleted list item.
API-get-user API-post-search low A task like 'find or look up the user by name' may route to API-post-search because it's the only 'search' tool, even though it searches page/database titles, while API-get-user needs an ID the agent may not have.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard