Public assessment

flipt-io/mcp-server-flipt (@flipt-io/mcp-server-flipt)

flipt-io-mcp-server-flipt · v0.1.0 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 92→94, token cost up 24%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

19.3 / 30

19.3 out of 30

03Economics

18.4 / 20

18.4 out of 20

04Discoverability

9.5 / 20

9.5 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

28 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`list_namespaces`	no_return_description	Lists all namespaces in the Flipt instance and returns the complete set of namespaces.
`create_namespace`	params_unexplained no_return_description	Creates a new namespace in the Flipt instance. 'key' is the unique identifier for the namespace, 'name' is its display name, and 'description' is an optional human-readable explanation. Returns the created namespace.
`update_namespace`	params_unexplained no_return_description	Updates an existing namespace identified by 'key', optionally changing its 'name' or 'description' fields. Returns the updated namespace.
`delete_namespace`	params_unexplained no_return_description	Deletes the namespace identified by 'key' from the Flipt instance along with its contents.
`list_flags`	params_unexplained no_return_description	Lists all flags in the namespace identified by 'namespaceKey' and returns the list of flags.
`get_flag`	params_unexplained no_return_description	Returns the flag identified by 'flagKey' within the namespace identified by 'namespaceKey'.
`create_flag`	params_unexplained no_return_description	Creates a flag in the namespace identified by 'namespaceKey'. 'key' is the flag's unique identifier, 'name' is its display name, 'description' is optional, 'enabled' sets whether the flag starts enabled, and 'type' (VARIANT_FLAG_TYPE or BOOLEAN_FLAG_TYPE) determines the flag kind. Returns the created flag.
`update_flag`	params_unexplained no_return_description	Updates an existing flag identified by 'key' in the namespace identified by 'namespaceKey', optionally changing its 'name', 'description', or 'enabled' state. Returns the updated flag.
`delete_flag`	params_unexplained no_return_description	Deletes the flag identified by 'key' from the namespace identified by 'namespaceKey'.
`toggle_flag`	params_unexplained no_return_description	Sets the enabled state of the flag identified by 'flagKey' in the namespace identified by 'namespaceKey' to the boolean value 'enabled'. Returns the updated flag.
`list_segments`	params_unexplained no_return_description	Lists all segments in the namespace identified by 'namespaceKey' and returns the list of segments.
`get_segment`	params_unexplained no_return_description	Returns the segment identified by 'segmentKey' in the namespace identified by 'namespaceKey'.
`create_segment`	params_unexplained no_return_description	Creates a segment in the namespace identified by 'namespaceKey'. 'key' is the segment's unique identifier, 'name' is its display name, 'description' is optional, and 'matchType' (ALL_MATCH_TYPE or ANY_MATCH_TYPE) controls whether constraints must all match or any must match. Returns the created segment.
`update_segment`	params_unexplained no_return_description	Updates the segment identified by 'key' in the namespace identified by 'namespaceKey', optionally changing its 'name', 'description', or 'matchType'. Returns the updated segment.
`delete_segment`	params_unexplained no_return_description	Deletes the segment identified by 'key' from the namespace identified by 'namespaceKey'.
`evaluate_boolean_flag`	params_unexplained no_return_description	Evaluates the boolean flag identified by 'flagKey' in the namespace identified by 'namespaceKey' for the entity identified by 'entityId', using the optional 'context' key-value pairs for targeting. Returns the boolean evaluation result along with any match reasons.
`evaluate_variant_flag`	params_unexplained no_return_description	Evaluates the variant flag identified by 'flagKey' in the namespace identified by 'namespaceKey' for the entity identified by 'entityId', using the optional 'context' key-value pairs for targeting. Returns the matched variant key and any match reasons.
`evaluate_batch`	params_unexplained no_return_description	Evaluates multiple flags in a single call. 'requests' is an array of evaluations, each specifying namespaceKey, flagKey, entityId, and an optional context object. Returns the evaluation result for each request.
`create_constraint`	params_unexplained no_return_description	Creates a constraint on the segment identified by 'segmentKey' in the namespace identified by 'namespaceKey'. 'type' is the comparison type, 'property' is the context property to compare, 'operator' is the comparison operator, 'value' is the comparison value, and 'description' is optional. Returns the created constraint.
`delete_constraint`	params_unexplained no_return_description	Deletes the constraint identified by 'constraintId' from the segment identified by 'segmentKey' in the namespace identified by 'namespaceKey'.
`create_variant`	params_unexplained no_return_description	Creates a variant for the flag identified by 'flagKey' in the namespace identified by 'namespaceKey'. 'key' is the variant's unique identifier, 'name' and 'description' are display metadata, and 'attachment' is optional string data. Returns the created variant.
`delete_variant`	params_unexplained no_return_description	Deletes the variant identified by 'variantId' from the flag identified by 'flagKey' in the namespace identified by 'namespaceKey'.
`create_rule`	params_unexplained no_return_description	Creates a rollout rule for the flag identified by 'flagKey' in the namespace identified by 'namespaceKey', targeting the segment identified by 'segmentKey'. 'rank' sets evaluation order and 'segmentOperator' (OR_SEGMENT_OPERATOR or AND_SEGMENT_OPERATOR) controls segment matching. Returns the created rule.
`delete_rule`	params_unexplained no_return_description	Deletes the rule identified by 'ruleId' from the flag identified by 'flagKey' in the namespace identified by 'namespaceKey'.
`create_distribution`	params_unexplained no_return_description	Creates a distribution that assigns the rollout percentage 'rollout' (0-100) of traffic to the variant identified by 'variantId' for the rule identified by 'ruleId', within the flag identified by 'flagKey' in the namespace identified by 'namespaceKey'. Returns the created distribution.
`delete_distribution`	params_unexplained no_return_description	Deletes a traffic distribution (a variant's allocation) from a specific rule of a flag, removing that variant's share of the rule's traffic. Use when you need to remove a variant bucket from an existing distribution rule. Parameters: namespaceKey (string) - the namespace that owns the flag; flagKey (string) - the flag whose rule contains the distribution; ruleId (string) - ID of the rule the distribution belongs to; distributionId (string) - ID of the distribution to delete; variantId (string) - the variant associated with the distribution. Returns nothing on success; throws an error if the distribution, rule, flag, or namespace does not exist.
`create_rollout`	params_unexplained no_return_description	Creates a new rollout for a flag, which controls how the flag resolves to a boolean value for a given evaluation context. A rollout is either segment-based (returns the given value when the context matches a segment) or threshold-based (returns the given value when a randomly generated percentage falls below the threshold). Parameters: namespaceKey (string) - the namespace that owns the flag; flagKey (string) - the flag the rollout applies to; rank (integer, >=1) - the rollout's evaluation order, where lower-ranked rollouts are evaluated first; description (string) - optional human-readable note about the rollout; segment (object) - segment-based rollout: segmentKey is the key of the segment to match and value is the flag value returned on match; threshold (object) - threshold-based rollout: percentage is 0-100 defining the rollout threshold and value is the flag value returned when the threshold is met. Returns the created rollout resource on success; throws an error if the flag or namespace does not exist or if inputs are invalid.
`delete_rollout`	params_unexplained no_return_description	Deletes an existing rollout from a flag, removing that rollout's rule from the flag's evaluation logic. Use when you want to stop a particular segment- or threshold-based rollout from applying to a flag. Parameters: namespaceKey (string) - the namespace that owns the flag; flagKey (string) - the flag whose rollout is being removed; rolloutId (string) - ID of the rollout to delete. Returns nothing on success; throws an error if the rollout, flag, or namespace does not exist.

Selection evidence

Confusable tool pairs.

6 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`evaluate_boolean_flag`	`evaluate_variant_flag`	high	Both tools have identical schemas (namespaceKey/flagKey/entityId/context) and only differ by the flag type adjective; a user saying 'evaluate flag X for user Y' without stating whether it is boolean or variant cannot be disambiguated from the names alone.
`update_flag`	`toggle_flag`	medium	Both can set a flag's enabled state; a request like 'turn flag X on' or 'set flag X to enabled' matches update_flag's enabled field and toggle_flag's enabled field equally, so an agent could pick the wrong mutating tool.
`create_distribution`	`create_rollout`	medium	Both model percentage-based rollout (a distribution's 'rollout' percent vs a rollout's 'threshold.percentage'); 'create a 50% rollout for this flag' plausibly maps the word rollout to either tool.
`delete_distribution`	`delete_rollout`	medium	Mirrors the create ambiguity: 'remove the rollout on flag X' could target either a distribution (which carries a rollout percentage) or a rollout entity, and both take namespaceKey/flagKey.
`create_constraint`	`create_rule`	medium	Both attach logic to a segment and share namespaceKey/segmentKey; users colloquially call constraints 'rules', so 'add a rule to segment S matching property X' could select create_rule instead of create_constraint or vice versa.
`delete_constraint`	`delete_rule`	medium	Same rule/constraint terminology overlap on deletion: 'delete the rule on segment S' is ambiguous between removing a segment constraint and removing a flag-segment rule, since both are referred to as rules.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard