0.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 15%, unconfirmed writes 0%→0%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
01Safety
02Legibility
18.0 / 30
03Economics
14.2 / 20
04Discoverability
10.4 / 20
Highest-impact fix
Estimated gain +30 pointsAdd explicit identity and permission preflight tools
Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.
Description evidence
Defects and rewrites.
64 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
list_websites |
name_restates_behavior no_return_description |
Returns a paginated list of all websites tracked in Umami, optionally filtered by a search query (query) and ordered by a specified field (orderBy, e.g. 'name' or 'domain'); page and pageSize control pagination (default 10). |
get_website |
name_restates_behavior no_return_description |
Returns the full details of a single website identified by its UUID (websiteId), including its name, domain, and configuration. |
create_website |
name_restates_behavior no_return_description |
Creates a new website in Umami to start tracking its traffic; requires a domain and display name, and optionally a unique shareId for public access; returns the created website record and its generated UUID. |
update_website |
name_restates_behavior no_return_description |
Updates the name, domain, and/or shareId of an existing website identified by websiteId; pass null for shareId to remove public share access; returns the updated website record. |
delete_website |
name_restates_behavior no_return_description |
Permanently removes the website identified by websiteId and its tracked analytics data from Umami; returns the result of the deletion. |
get_active_visitors |
name_restates_behavior no_return_description |
Returns the real-time number of visitors currently active on the website identified by websiteId (usable for showing live visitor counts). |
reset_website |
no_return_description |
Reset a website by removing all of its analytics data; this operation is irreversible. Returns the result of the reset operation. |
transfer_website |
name_restates_behavior no_return_description |
Transfers ownership of the website identified by websiteId to the user identified by userId, granting that user control of the website; returns the result of the transfer. |
get_website_reports |
name_restates_behavior no_return_description |
Returns all analytics reports associated with the website identified by websiteId. |
get_stats |
no_return_description |
Returns summary statistics for the website identified by websiteId within the required startAt/endAt timestamp window (milliseconds), including pageviews, visitors, visits, bounces, and totaltime; optionally filtered by url or referrer. |
get_pageviews |
name_restates_behavior no_return_description |
Returns time-series pageview and session counts for a website within startAt/endAt, grouped by the given unit (hour/day/week/month/year) and timezone, optionally filtered by url or referrer; yields per-interval counts for the requested range. |
get_metrics |
name_restates_behavior no_return_description |
Returns aggregated metric counts for a website within startAt/endAt, grouped by the requested metric type (e.g. url, browser, country, device, os, event); optional url/referrer filters apply and limit caps the number of results (default 500). |
get_events |
name_restates_behavior no_return_description |
Returns time-series event data (event counts) for a website within startAt/endAt, grouped by unit (hour/day/week/month/year) and timezone, optionally filtered by url; yields per-interval event counts. |
get_sessions |
name_restates_behavior params_unexplained no_return_description |
Returns a paginated list of sessions recorded for a website within startAt/endAt, optionally filtered by a search query, sorted by the specified session attribute (orderBy), with page and pageSize controlling pagination; yields per-session details. |
get_daterange |
no_return_description |
Returns the earliest and latest timestamps of analytics data available for the website identified by websiteId, defining the valid startAt/endAt window. |
get_event_series |
name_restates_behavior no_return_description |
Returns time-series metrics for an event on a website within startAt/endAt, grouped by unit (hour/day/week/month/year) and timezone, optionally filtered by url or eventName; yields per-interval event metrics. |
get_session_stats |
no_return_description |
Returns summarized session statistics (total sessions, unique visitors, etc.) for a website within startAt/endAt, optionally filtered by url or referrer. |
get_sessions_weekly |
name_restates_behavior no_return_description |
Returns weekly aggregated session data for a website within the startAt/endAt timestamp window, grouped by week. |
get_session |
name_restates_behavior no_return_description |
Returns the full details of a single session, identified by websiteId (website UUID) and sessionId (session UUID). |
get_session_activity |
name_restates_behavior no_return_description |
Returns the activity log for the session identified by websiteId and sessionId, listing the pages visited and events fired during that session. |
get_session_properties |
name_restates_behavior no_return_description |
Returns the custom properties (key-value attributes) attached to the session identified by websiteId and sessionId. |
get_session_data_properties |
name_restates_behavior no_return_description |
Returns the names and data types of session data properties collected for a website within the startAt/endAt timestamp window. |
get_session_data_values |
name_restates_behavior no_return_description |
Returns aggregated counts for session data property values for a website within startAt/endAt, optionally filtered to a single propertyName; aids in building property filter options. |
send_event |
name_restates_behavior no_return_description |
Sends a custom event or pageview to Umami for the website identified by websiteId (mapped to the payload 'website' field), useful for server-side tracking; requires hostname and url, accepts optional eventName/eventData (or omit eventName for a pageview) plus referrer, language, and title; returns the result of the tracking request. |
get_event_values |
name_restates_behavior no_return_description |
Returns the distinct event or session property values collected for a website within the startAt/endAt timestamp window; useful for building filter dropdowns. |
get_event_data_events |
name_restates_behavior no_return_description |
Retrieve custom event names and their occurrence counts for a website within a time range. Use this to see which custom events fire most often. Returns a list of event names with counts. Optionally filter to a single event name. |
get_event_data_fields |
name_restates_behavior no_return_description |
Retrieve the property keys and data types attached to a website's custom event data within a time range. Use this to discover what fields are available for filtering or aggregation. Returns a list of property keys with their data types. Optionally filter to a single event name. |
get_event_data_values |
name_restates_behavior no_return_description |
Retrieve aggregated counts for the values of a specific event-data property for a website within a time range. Use this to see the value distribution of a property. Returns per-value event counts. Supply eventName and propertyName to target the property. |
get_event_data_stats |
name_restates_behavior no_return_description |
Retrieve summary statistics for a website's custom event data within a time range. Use this to get aggregate counts such as total events and distinct values. Returns summary counts for the events. Optionally filter to a single event name. |
batch_events |
no_return_description |
Send multiple custom events or pageviews in a single request to reduce network overhead. Use this to report analytics data in bulk. Returns the result of the ingestion request. Each event requires websiteId, hostname, and url; omit eventName for pageviews. |
list_reports |
name_restates_behavior params_unexplained no_return_description |
Retrieve a paginated list of all saved reports. Use this to browse available reports before opening one. Returns an array of reports with their metadata. Optionally paginate with page and pageSize, and orderBy specifies the field used to sort the results. |
get_report |
name_restates_behavior no_return_description |
Retrieve the full details of a specific saved report by its UUID, including its configuration and parameters. Use this to load a report for viewing or editing. Returns the report's metadata and configuration for the given reportId. |
create_report |
name_restates_behavior no_return_description |
Create and persist a new saved report for a website, then return the created report with its generated UUID. Use this when defining a reusable report configuration. Configure it with name, type, and report-specific parameters. |
update_report |
name_restates_behavior no_return_description |
Modify an existing saved report's name, type, description, or parameters in place. Use this when a report's configuration changes. Returns the updated report. The reportId identifies which report to update. |
delete_report |
name_restates_behavior no_return_description |
Permanently remove a saved report by its UUID. Use this to clean up reports that are no longer needed. Returns a confirmation that the report was deleted. |
list_users |
name_restates_behavior no_return_description |
Retrieve a paginated, searchable list of all users in the instance, for admin use. Use this to review accounts. Returns an array of user records. Optionally filter by query, paginate with page/pageSize, and order results by a field via orderBy. |
create_user |
name_restates_behavior no_return_description |
Create a new user account with a username and password, for admin use. Use this to provision access for new members. Returns the created user record. Optionally assign a role ('admin' or 'user', default 'user'). |
get_user |
name_restates_behavior no_return_description |
Retrieve the details of a specific user account by UUID, for admin use. Use this to inspect a single account. Returns the user record for the given userId. |
update_user |
name_restates_behavior no_return_description |
Change a user's username, password, or role, for admin use. Use this to manage account credentials and permissions. Returns the updated user record. Only the supplied fields are changed for the given userId. |
delete_user |
name_restates_behavior no_return_description |
Permanently remove a user account by its UUID, for admin use. Use this to revoke access. Returns a confirmation that the user was deleted. |
get_user_websites |
no_return_description |
Retrieve the list of websites a specific user has access to, for admin use. Use this to audit a user's access. Returns website records with pagination. Optionally filter by query and paginate with page/pageSize for the given userId. |
get_user_usage |
name_restates_behavior no_return_description |
Retrieve usage statistics for a specific user within a time range, for admin use. Use this to review a user's activity volume. Returns usage counts between startAt and endAt for the given userId. |
get_user_teams |
no_return_description |
Retrieve the list of teams a specific user belongs to, for admin use. Use this to see a user's team memberships. Returns team records with pagination. Optionally filter by query and paginate with page/pageSize for the given userId. |
list_teams |
name_restates_behavior no_return_description |
Retrieve a paginated, searchable list of all teams. Use this to browse teams in the instance. Returns an array of team records. Optionally filter by query, paginate with page/pageSize, and order results by a field via orderBy. |
create_team |
name_restates_behavior no_return_description |
Create a new team with a name. Use this to set up a group that can share website access. Returns the created team record including its UUID. |
get_team |
name_restates_behavior no_return_description |
Retrieve the details of a specific team by its UUID. Use this to inspect a team before editing. Returns the team record for the given teamId. |
update_team |
name_restates_behavior no_return_description |
Rename an existing team. Use this to change a team's display name. Returns the updated team record for the given teamId. |
delete_team |
name_restates_behavior no_return_description |
Permanently remove a team by its UUID. Use this to disband a team. Returns a confirmation that the team was deleted. |
join_team |
name_restates_behavior no_return_description |
Join an existing team by entering its access or invite code, which adds the current user to that team. Returns the team record for the team that was joined. |
list_team_users |
no_return_description |
Returns a paginated list of members of the team identified by teamId, including each member's user details and role. Use page and pageSize to control pagination and query to filter members by name or username. |
add_team_user |
name_restates_behavior no_return_description |
Adds an existing user, identified by userId, to the team identified by teamId, assigning them the given role ('team-owner' or 'team-member'). Returns the created team membership record. |
update_team_user |
no_return_description |
Changes the role of the existing team member identified by userId within the team identified by teamId to the new role ('team-owner' or 'team-member'). Returns the updated membership record with the new role. |
get_team_user |
no_return_description |
Returns detailed information for a single team member identified by userId within the team identified by teamId, including their user profile and role. |
remove_team_user |
name_restates_behavior no_return_description |
Removes the user identified by userId from the team identified by teamId, revoking their access to the team's websites. Returns a confirmation of the removal. |
list_team_websites |
name_restates_behavior no_return_description |
Returns a paginated list of websites belonging to the team identified by teamId. Use page and pageSize to control pagination and query to filter websites by name or domain. |
add_team_website |
name_restates_behavior no_return_description |
Grants the team identified by teamId access to the website identified by websiteId so team members can view its analytics. Returns the created team-website association record. |
remove_team_website |
name_restates_behavior no_return_description |
Revokes the team identified by teamId access to the website identified by websiteId. Returns confirmation that the website was removed from the team. |
get_me |
no_return_description |
Returns the profile information for the currently authenticated user, including their name, username, email, and role. |
get_my_websites |
no_return_description |
Returns a paginated list of websites belonging to the current user. Use page and pageSize to control pagination and query to filter websites by name or domain. |
get_my_teams |
no_return_description |
Returns a paginated list of teams the current user belongs to. Use page and pageSize to control pagination and query to filter teams by name. |
update_my_password |
params_unexplained no_return_description |
Changes the current user's password: currentPassword is the user's existing password, required to verify their identity, and newPassword is the replacement password to set. Returns confirmation once the password has been updated. |
verify_auth |
no_return_description |
Checks whether the current authentication token is valid and returns a status indicating whether it can be used to authenticate subsequent requests. |
get_share |
name_restates_behavior no_return_description |
Returns analytics data for a publicly shared website identified by its share ID. The share ID itself grants access, so no authentication is required to retrieve the shared data. |
heartbeat |
no_return_description |
Returns the server's health and version status to confirm that the Umami server is running and reachable. Useful as a connectivity and readiness check before making other calls. |
Selection evidence
Confusable tool pairs.
17 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
get_sessions |
get_session |
medium | Singular/plural names sharing the 'session' token make a task like 'get the session data for this website' plausibly map to either the session list or the single-session lookup |
get_user_teams |
get_team_user |
medium | Mirrored team/user word order means a task like 'show the user's team details' could pick the user's team list instead of the specific team-member lookup |
get_event_data_events |
get_event_data_fields |
medium | Identical 'get event data' prefix and shared eventName input make a vague task like 'what is in my event data' ambiguous between event names/counts and property fields |
get_event_data_events |
get_event_data_stats |
medium | Both return event counts, so 'how many events do I have' could select either the per-event count list or the summary stats tool |
get_event_data_events |
get_event_data_values |
medium | Same websiteId/startAt/endAt/eventName inputs and both return counts, so 'get the counts for my events' fits either tool |
get_event_values |
get_event_data_values |
high | Names differ only by 'data' and both return event/session property values, so 'get the event values for this site' matches both almost exactly |
get_session_properties |
get_session_data_properties |
medium | Names differ only by 'data'; 'get session properties' could target a specific session's custom properties or the website-level session property schema |
get_sessions |
get_sessions_weekly |
medium | A task like 'get the sessions this week' contains both 'sessions' and 'week', matching both the general session list and the weekly session tool |
get_events |
get_event_series |
high | Both take the same websiteId/startAt/endAt/unit inputs and both return events over time, so any task about event trends or event time series is genuinely ambiguous |
get_events |
get_event_data_events |
medium | 'Get events' literally matches both names; only the time-grouping requirement and 'custom' wording in the descriptions separate a query about event counts or event timing |
get_stats |
get_session_stats |
medium | Both are summary statistics with visitor/session counts, so 'get the stats for this website' could return the general pageview stats or the session-focused stats |
get_website_reports |
get_report |
medium | Both return report data but are keyed by websiteId vs reportId, so 'show the report for this website' could pick the per-site report list or a specific saved report |
update_user |
update_team_user |
medium | Both update a user's role, so 'change user X's role' is ambiguous between the global user role and the team membership role |
get_event_data_fields |
get_event_data_values |
medium | Both expose event property data, one as field names/types and the other as aggregated counts, so 'show my event property data' is ambiguous between schema and values |
get_event_data_values |
get_event_data_stats |
medium | Both return aggregated counts for event data, so 'get counts for my event data' could select the per-property values or the summary stats tool |
get_session_data_properties |
get_session_data_values |
medium | Shared 'get session data' prefix means a task mixing properties and values, such as 'what session properties and their values exist', could map to the wrong member |
get_session_stats |
get_sessions_weekly |
medium | A phrase like 'get weekly session stats' mixes the weekly aggregation tool and the session statistics tool, and both accept websiteId plus a time range |
Compare the field