Public assessment

tamasPetki/HeadlessTracker (headless-tracker)

headless-tracker · v1.0.17 · scanned Aug 3, 2026

What changed in the harness

Selection accuracy 88→88, token cost down 2%, unconfirmed writes 0%→0%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

0.0 / 30

0.0 out of 30

02Legibility

25.7 / 30

25.7 out of 30

03Economics

15.0 / 20

15.0 out of 20

04Discoverability

14.5 / 20

14.5 out of 20

Highest-impact fix

Estimated gain +30 points

Add explicit identity and permission preflight tools

Expose machine-readable principal/tenant confirmation and a non-mutating permission check so agents can verify both before destructive actions.

Description evidence

Defects and rewrites.

4 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`refresh_data`	no_return_description	Invalidates the PORTFOLIO TRACKER cache (Bybit/Binance/MetaMask/Polymarket/Solana/Hyperliquid holdings + transactions) so the next data call fetches fresh values from upstream APIs. Use this when the user says 'refresh my portfolio', 'update my holdings', 'my balances look stale', 'force an update', or otherwise implies tracker data may be outdated, then follow up with get_holdings or another data tool — this tool only marks the cache stale, it does not fetch on its own. Optionally scope to a single connector (bybit, binance, metamask, polymarket, solana, hyperliquid); omit to invalidate every tracker connector. Returns a confirmation of success listing which connectors were invalidated. Do not use for refreshing webpages, OAuth tokens, browser cache, or data from non-tracker MCP servers.
`add_wallet_address`	no_return_description	Adds another wallet address to an existing MetaMask or Solana account so the tracker covers it too. Use when the user asks 'add another wallet', 'track a second address', 'add MetaMask address', 'add Solana address'. The new address inherits the parent account's settings (Etherscan key + chains for MetaMask; RPC URL for Solana). Wallet addresses are public on-chain identifiers, so no new secrets are stored — it just updates the keychain entry's addresses[] field. Returns confirmation of the update, including the account id and the now-tracked addresses.
`add_custom_token`	no_return_description	Adds a project-specific ERC-20 token to a MetaMask account's tracked-tokens list so its balance appears in holdings. The bundled common tokens (USDC, USDT, WETH, WBTC, LINK, DAI) are tracked by default; use this for extra tokens like project or governance tokens. Use when the user asks 'track ARB token', 'add UNI to my wallet', 'monitor a custom ERC-20'. Token data is public on-chain (contract + symbol + decimals) — no secrets or keychain involvement. Returns confirmation of the added token, including the account id, chain id, contract, symbol, and decimals.
`list_custom_tokens`	no_return_description	Lists the custom ERC-20 tokens tracked by MetaMask accounts, returning for each token its symbol, contract address, chain, and decimals so users can see which project tokens they monitor beyond the bundled defaults. Public data; no secrets involved. Use when the user asks 'which custom tokens am I tracking', 'show my project tokens'. Optionally pass account_id to filter to one MetaMask account; omit to return tracked custom tokens across all accounts.

Selection evidence

Confusable tool pairs.

7 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get_holdings`	`get_allocations`	medium	Both describe portfolio snapshots with USD values; queries like 'what's my biggest holding/position', 'portfolio breakdown', or 'how is my portfolio split' sit on the boundary since get_holdings covers 'current positions'/'show my portfolio' while get_allocations owns '%', 'biggest position', and 'concentration'. The overlap is so known that get_holdings itself defers ranked splits to get_allocations, yet 'biggest holding' phrasing can still route to get_holdings.
`get_pnl`	`get_polymarket_positions`	medium	A Polymarket-scoped profit question ('am I up on Polymarket', 'what's my Polymarket P&L/profit') is claimed by both: get_polymarket_positions returns cashPnl/avgCost/currentValue and lists 'how am I doing on Polymarket' as a trigger, while get_pnl covers all configured accounts including Polymarket with realized/unrealized PnL.
`get_holdings`	`get_polymarket_positions`	low	Definitions cross-reference (Polymarket-specific → get_polymarket_positions; mixed portfolio → get_holdings), but 'how much do I have in Polymarket / prediction-market holdings and balance' can still route either way because both return per-position USD values.
`list_accounts`	`list_custom_tokens`	low	Both are listing tools keyed by 'tracking' vocabulary — list_accounts ('which wallets/addresses am I tracking') and list_custom_tokens ('which custom tokens am I tracking'). A generic 'what am I tracking' or 'show what's tracked' request is ambiguous between tracker accounts and tracked ERC-20 tokens.
`get_polymarket_positions`	`get_allocations`	low	'Polymarket allocation/breakdown by market or asset class' could be served by either: get_allocations groups by asset_class/symbol with top-N concentration, while get_polymarket_positions groups by event and returns currentValue; neither description routes such queries to the other.
`get_polymarket_positions`	`get_transactions`	low	'Show my Polymarket bets / betting history' straddles current positions (get_polymarket_positions trigger: 'what bets do I have') and trade history (get_transactions covers Polymarket BUY/SELL trades); surface wording like 'bets' vs 'history' is decisive but the phrasing is genuinely ambiguous.
`render_dashboard`	`render_settings`	low	Both render live MCP App panels for the same headless-tracker server, so a vague 'open the tracker panel / portfolio tracker UI' could select either; render_settings also claims 'show my tracker accounts' and 'add a custom ERC-20 token' as triggers, overlapping the dashboard's portfolio-view scope.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard