Public assessment

spranab/saga-mcp (saga-mcp)

saga-mcp · v1.0.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 100→100, token cost up 18%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

21.5 / 30

21.5 out of 30

03Economics

17.3 / 20

17.3 out of 20

04Discoverability

12.0 / 20

12.0 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

25 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`project_create`	name_restates_behavior no_return_description	Create a new project, the top-level container that holds all related epics, tasks, notes, and comments. Set status (active/on_hold/completed/archived) to mark its lifecycle state and tags to categorize it. Returns the created project including its ID and configured fields.
`project_list`	name_restates_behavior no_return_description	Return every project, each with its epic/task counts and completion percentage. Optionally pass status (active/on_hold/completed/archived) to narrow results to a single lifecycle state. Returns a list of projects, empty if none match.
`project_update`	params_unexplained no_return_description	Update an existing project by ID, changing only the provided fields. Use name and description to edit its details, status (active/on_hold/completed/archived; set to "archived" to soft-delete), and tags to replace its categorization. Returns the updated project.
`epic_create`	params_unexplained no_return_description	Create an epic within a project to group related tasks into a feature or workstream. Set status (planned/in_progress/completed/cancelled) and priority (low/medium/high/critical) to track lifecycle and importance, tags to categorize, and branch to scope the epic to a git branch (pass "current" to auto-detect, or omit/empty string for a branch-agnostic epic). Returns the created epic with its ID.
`epic_list`	params_unexplained no_return_description	Return the epics belonging to the given project_id, each with task counts and completion stats. Optionally filter by status (planned/in_progress/completed/cancelled), priority (low/medium/high/critical), or branch: pass "current" to auto-detect the active git branch, an empty string to list only branch-agnostic epics, or omit branch to include all. Returns a list of epics, empty if none match.
`epic_update`	params_unexplained no_return_description	Update an existing epic by ID, changing only the provided fields. Use name and description for edits, status (planned/in_progress/completed/cancelled; set to "cancelled" to soft-delete), priority (low/medium/high/critical), sort_order to control its position, tags to replace categorization, and branch to re-scope it ("current" auto-detects the active branch; empty string clears to branch-agnostic). Returns the updated epic.
`task_create`	name_restates_behavior params_unexplained no_return_description	Create a task inside an epic, the primary unit of work in the tracker. Set status (todo/in_progress/review/done/blocked) and priority (low/medium/high/critical) to track progress and importance, estimated_hours for scheduling effort, and tags for categorization; optionally assign it (assigned_to), set a due date, link source code via source_ref, or declare dependencies with depends_on. Returns the created task with its ID.
`task_list`	params_unexplained no_return_description	Return tasks matching the optional filters, each including subtask counts and dependency info. Omit epic_id to list across all epics; filter by status (todo/in_progress/review/done/blocked), priority (low/medium/high/critical), assigned_to, or tag; restrict by branch ("current" limits to epics on the active git branch; empty string to branch-agnostic only); sort via sort_by and cap results with limit. Returns a list of tasks, empty if none match.
`task_get`	name_restates_behavior no_return_description	Fetch a single task by ID with its full record in one response: all subtasks, related notes, comments, and dependency information. Returns the complete task object.
`task_update`	name_restates_behavior params_unexplained no_return_description	Update an existing task by ID, changing only the provided fields. Use title and description for edits, status (todo/in_progress/review/done/blocked; transitions are automatically logged in the activity log), priority (low/medium/high/critical), assigned_to, estimated_hours and actual_hours for effort tracking, due_date, sort_order, tags, source_ref, and depends_on (replaces the existing dependency set). Returns the updated task.
`subtask_create`	params_unexplained no_return_description	Create one or more subtasks (checklist items) under a task. Pass titles as a single string for one subtask or an array of strings to batch-create several in a single call. Returns the created subtasks with their IDs.
`subtask_update`	name_restates_behavior params_unexplained no_return_description	Modify an existing subtask by ID, changing only the provided fields: title to rename it, status among todo/in_progress/done to track progress, or sort_order to reorder it within its parent task. Returns the updated subtask.
`subtask_delete`	name_restates_behavior params_unexplained no_return_description	Permanently delete one or more subtasks by ID. Pass a single integer or an array of integers to remove multiple in one call. Returns confirmation of the deletion.
`note_save`	params_unexplained	Create a new note or update an existing one: pass id to update, otherwise a new note is created. Notes capture decisions, context, progress, meeting notes, blockers, technical details, or release info. Set note_type (general/decision/context/meeting/technical/blocker/progress/release) to categorize the note, related_entity_type plus related_entity_id to link it to a specific project, epic, or task, and tags for additional categorization.
`note_list`	params_unexplained	List notes sorted most recent first, optionally filtered by note_type (general/decision/context/meeting/technical/blocker/progress/release), related_entity_type plus related_entity_id to target a specific project, epic, or task, a single tag, or limit to cap how many are returned. Returns a list of matching notes, empty if none.
`note_search`	name_restates_behavior params_unexplained no_return_description	Search across note titles and content for the given keyword query. Optionally narrow results to a note_type (general/decision/context/meeting/technical/blocker/progress/release) and cap the number of results with limit (default 20). Returns the matching notes.
`note_delete`	name_restates_behavior no_return_description	Permanently remove the note with the given id. Returns confirmation that the note was deleted.
`comment_add`	params_unexplained no_return_description	Add a comment to a task, appending it to the task's chronological discussion thread so decision context is preserved across sessions. Optionally set author to attribute the comment. Returns the created comment.
`comment_list`	name_restates_behavior params_unexplained no_return_description	Return the complete comment thread for the given task_id, ordered chronologically (oldest to newest). Returns a list of comments, empty if the task has none.
`template_create`	no_return_description	Create a reusable task template that defines a set of tasks to instantiate into any epic later via template_apply. Each task definition may specify a title, description, priority, estimated_hours, and tags; use {variable} placeholders in titles and descriptions for values filled at apply time. The name must be unique. Returns the created template.
`template_list`	name_restates_behavior no_return_description	Return all saved task templates that can be applied to epics, including each template's name, description, and task definitions. Returns a list of templates, empty if none exist.
`template_delete`	name_restates_behavior no_return_description	Permanently remove the task template with the given id so it can no longer be listed or applied to epics. Returns confirmation of the deletion.
`activity_log`	params_unexplained no_return_description	View the activity log showing what changed and when. Useful for understanding recent progress or reviewing what happened since the last session. Returns a list of activity entries, each with a timestamp, the entity affected, and the action performed. Filter by entity_type and/or entity_id to narrow to one kind of item or a single entity, by action to see only created/updated/deleted/status_changed events, or by since (ISO 8601 datetime) to see only activity after that time. limit caps how many entries are returned (default 50).
`task_batch_update`	params_unexplained no_return_description	Update multiple tasks in one call by setting their status, priority, or assignee. Useful for changing status of several tasks (e.g., mark 3 tasks as done) or reassigning tasks. status sets the workflow state (todo, in_progress, review, done, blocked), priority sets importance (low, medium, high, critical), and assigned_to sets the owner (pass a user identifier to (re)assign). ids lists the tasks to update. Returns a summary of the updated tasks, including any that could not be updated.
`tracker_import`	no_return_description	Import a project from JSON (matching tracker_export format). Creates all entities with new IDs and remaps references so cross-references remain valid under the new IDs. Uses a transaction for atomicity, so either the whole project is imported or nothing changes. Returns a summary of the import, including the new project ID and counts of the entities created.

Selection evidence

Confusable tool pairs.

3 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`task_update`	`task_batch_update`	high	Both modify task status/properties and share the 'task update' phrasing. A user request like 'mark these tasks as done' or 'update the tasks' could lead the agent to task_update (single-ID input) when multiple tasks change, or to the batch tool when only one is intended, since neither description forces the count.
`note_search`	`tracker_search`	medium	Both are keyword searches, and tracker_search explicitly includes notes ('Search across ALL entities ... projects, epics, tasks, notes'). A request like 'search for deployment issues' over an unspecified scope is ambiguous: note_search is the narrow note-only search while tracker_search also satisfies it by returning notes.
`note_list`	`note_search`	medium	Both retrieve notes, one by structured filters and one by keyword. A request like 'find my notes about the onboarding flow' could map to note_list (filter by tag/type) or note_search (query), since neither description explicitly disambiguates 'list/filter' versus 'keyword query' for a fuzzy find request.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard