Public assessment

spranab/saga-mcp (saga-mcp)

saga-mcp · v1.0.0 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 8%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

21.0 / 30

21.0 out of 30

03Economics

17.0 / 20

17.0 out of 20

04Discoverability

11.0 / 20

11.0 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

23 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`project_create`	name_restates_behavior no_return_description	Creates a new project as a top-level container for epics and tasks, with optional description, status, and tags. Returns the created project's ID and fields.
`project_update`	params_unexplained no_return_description	Updates an existing project's name, description, status, or tags; only included fields are changed. Set status to "archived" to soft-delete the project. Returns the updated project record.
`epic_create`	params_unexplained no_return_description	Creates an epic within a project to group related tasks into a feature or workstream, optionally scoped to a git branch (pass "current" to auto-detect, or omit for a branch-agnostic epic). status defaults to "planned", priority to "medium"; tags are free-form labels for filtering. Returns the created epic record including its ID.
`epic_list`	params_unexplained	Lists epics for a project, including task counts and completion stats for each. Optionally filter by status (planned/in_progress/completed/cancelled), priority (low/medium/high/critical), or git branch — pass branch="current" to auto-detect the active branch, or empty string to list only branch-agnostic epics.
`epic_update`	params_unexplained no_return_description	Updates an existing epic's name, description, status, priority, sort order, branch scope, or tags; only included fields are changed. Set status to "cancelled" to soft-delete. Pass branch="current" to pin to the active git branch, or empty string to clear it. Returns the updated epic record.
`task_create`	name_restates_behavior params_unexplained no_return_description	Creates a task within an epic, the primary unit of trackable work, with optional assignee, hour estimate, due date, source-code link, dependencies, and tags. status defaults to "todo" and priority to "medium"; tags are free-form labels for filtering. Returns the created task record including its ID.
`task_list`	params_unexplained	Lists tasks with optional filters, including subtask counts and dependency info. If no epic_id is given, lists across all epics. Filter by status (todo/in_progress/review/done/blocked) or priority (low/medium/high/critical); pass branch="current" to restrict to tasks whose epic is scoped to the active git branch.
`task_update`	params_unexplained no_return_description	Updates an existing task's title, description, status, priority, assignee, hour estimates, due date, source-code link, dependencies, sort order, or tags; only included fields are changed. Status transitions are automatically recorded in the activity log. Returns the updated task record.
`subtask_create`	no_return_description	Creates one or more subtasks (checklist items) under a task, accepting either a single title string or an array of titles for batch creation. Returns the created subtask record(s) including their IDs.
`subtask_update`	name_restates_behavior params_unexplained no_return_description	Updates an existing subtask's title, status (todo/in_progress/done), or sort order within its task's checklist; only included fields are changed. Returns the updated subtask record.
`subtask_delete`	no_return_description	Deletes one or more subtasks, accepting either a single ID or an array of IDs. Returns confirmation of the deleted subtask ID(s).
`note_save`	params_unexplained no_return_description	Creates a new note, or updates an existing one when "id" is provided. Notes capture decisions, context, meeting notes, blockers, technical details, progress, or release info, and can be tagged and linked to a project, epic, or task via related_entity_type/related_entity_id. note_type categorizes the note (general/decision/context/meeting/technical/blocker/progress/release; defaults to general). Returns the created or updated note record.
`note_list`	params_unexplained	Lists notes with optional filters, returned sorted by most recent first. Filter by note_type (general/decision/context/meeting/technical/blocker/progress/release), by related_entity_type/related_entity_id to scope to a project, epic, or task, or by tag; limit caps the number of results (default 30).
`note_search`	name_restates_behavior params_unexplained no_return_description	Searches note titles and content for a keyword match, optionally scoped to a note_type (general/decision/context/meeting/technical/blocker/progress/release) and capped at limit results (default 20). Returns matching note records.
`note_delete`	name_restates_behavior no_return_description	Deletes a note by its ID. Returns confirmation that the note was deleted.
`comment_add`	no_return_description	Adds a comment to a task, creating a chronological discussion thread useful for leaving breadcrumbs across sessions; author is optional. Returns the created comment record.
`template_create`	no_return_description	Creates a reusable task template that defines a set of tasks — using {variable} placeholders for dynamic values — which can later be instantiated into any epic via template_apply. Returns the created template record including its ID.
`template_list`	name_restates_behavior no_return_description	Lists all task templates available for instantiation into epics via template_apply. Returns each template's ID, name, description, and task definitions.
`template_apply`	no_return_description	Applies a template to create its defined tasks in an epic, substituting {variable} placeholders with the provided values. Returns the list of newly created task records.
`template_delete`	name_restates_behavior no_return_description	Deletes a task template by its ID. Returns confirmation that the template was deleted.
`activity_log`	params_unexplained	Views the activity log, showing what changed and when — useful for understanding recent progress or reviewing what happened since the last session. Filter by entity_type, entity_id, action, or since a given time; limit caps the number of entries returned (default 50).
`task_batch_update`	params_unexplained no_return_description	Updates status, priority, or assignee on multiple tasks at once by ID — e.g., mark several tasks as done, or reassign a batch of tasks. Returns the updated task records.
`tracker_import`	no_return_description	Imports a project from JSON in tracker_export format, creating all entities with new IDs and remapping references, executed as a single atomic transaction. Returns the newly created top-level project record.

Selection evidence

Confusable tool pairs.

6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`task_update`	`task_batch_update`	high	A request like 'mark tasks 12, 15, and 20 as done' could be fulfilled by looping task_update calls or by a single task_batch_update call; the tools overlap functionally (batch is a superset of single-task update) and the phrasing rarely signals which the caller intends.
`note_search`	`tracker_search`	medium	A vague instruction like 'search for anything about the auth migration' could map to note_search (notes only) or tracker_search (all entities including notes); without an explicit 'search notes' vs 'search everything' cue, an agent may pick the narrower or broader tool incorrectly.
`template_create`	`template_apply`	medium	Phrasing like 'create the standard onboarding tasks in this epic' could be misread as defining a new template (template_create) rather than instantiating an existing one into the epic (template_apply), since both involve 'creating tasks' language.
`note_list`	`note_search`	medium	A request such as 'find notes about the outage' could be served by note_search (keyword search) or misrouted to note_list with a tag/type filter, since both retrieve notes and the schemas don't make the retrieval mechanism obvious from a natural task description.
`epic_create`	`task_create`	low	A generic instruction like 'add a new item to track the payments feature' doesn't inherently specify whether it's a feature-level grouping (epic) or an individual unit of work (task), risking selection of the wrong granularity.
`tracker_dashboard`	`tracker_init`	low	Instructions like 'get this project set up and show me where things stand' could plausibly trigger tracker_init (which also returns existing project info if already initialized) instead of tracker_dashboard, or vice versa, since both can serve as an entry point for starting work on a project.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard