Public assessment

meacheal-ai/mrc-data (mrc-data)

mrc-data · v2.2.2 · scanned Aug 2, 2026

What changed in the harness

Selection accuracy 98→100, token cost down 0%, unconfirmed writes 100%→100%.

Category breakdown

Where the score comes from.

Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.

01Safety

30.0 / 30

30.0 out of 30

02Legibility

27.3 / 30

27.3 out of 30

03Economics

8.3 / 20

8.3 out of 20

04Discoverability

14.1 / 20

14.1 out of 20

Highest-impact fix

Estimated gain +12 points

Reduce the tool-definition and task token payload

Consolidate overlapping tools and trim repetitive schema text while preserving behavior, parameter, and return semantics.

Description evidence

Defects and rewrites.

1 defect found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.

Tool	Defect types	Suggested rewrite
`search_suppliers`	params_unexplained	Search verified Chinese apparel manufacturers, apparel factories, and clothing suppliers. USE WHEN user asks: - "find me a clothing manufacturer in China / Guangdong / Zhejiang" - "who makes [t-shirts / suits / denim / activewear] in China" - "I need a BSCI / OEKO-TEX certified apparel factory" - "looking for OEM / ODM apparel supplier with MOQ < N" - "find factories with production capacity > N pieces/month" - "list factories that export to the US / EU / Japan" - "show me trading companies in Yiwu / Shenzhen / Shanghai" - "which suppliers in [province] make [product]" (follow-up drill-down) - "give me another page of suppliers" (pagination via offset) - "who can produce knit tops under 300 MOQ" - "search by company name 新鑫 / Xinxin / Texhong" - "find workshop-scale suppliers for small batch sampling" - "搜供应商 / 找服装厂 / 找制衣厂 / 找代工厂 / 找外贸公司" - "帮我在[省份]找[品类]工厂，产能至少 N 件/月" Filters: province, city (exact city name in English or Chinese, e.g. Guangzhou, Shenzhen, Yiwu — narrows results within a province and can be combined with it or used alone), factory type (factory/trading_company/workshop), product category, minimum monthly capacity, compliance status, quality score. Returns paginated supplier list with company name, location, monthly capacity (lab-verified), compliance, quality score. WORKFLOW: Primary entry point for supplier discovery. search_suppliers → get_supplier_detail (for full 60+ field profile) OR compare_suppliers (side-by-side for up to 10 IDs) OR find_alternatives (diversify the pool) OR check_compliance (verify export readiness) OR get_supplier_fabrics (see their fabric catalog). RETURNS: { has_more: boolean, available_dimensions: string[], data: [{ supplier_id, company_name_cn, company_name_en, type, province, city, product_types, quality_score, verified_dims: "5/8", coverage_pct }] } EXAMPLES: • User: "Find BSCI-certified denim factories in Guangdong with MOQ under 500" → search_suppliers({ province: "Guangdong", product_type: "denim", compliance_status: "compliant", limit: 10 }) • User: "Who makes activewear for Lululemon in China?" → search_suppliers({ product_type: "activewear" }) — then filter results by client brand in get_supplier_detail • User: "我要在浙江找做牛仔的工厂，产能大于 10 万件" → search_suppliers({ province: "Zhejiang", product_type: "denim", min_capacity: 100000 }) • User: "Show me the next 10 trading companies in Yiwu" → search_suppliers({ city: "Yiwu", type: "trading_company", limit: 10, offset: 10 }) ERRORS & SELF-CORRECTION: • Empty data array → try these in order: (1) remove min_capacity filter, (2) drop city but keep province, (3) broaden product_type to parent category (e.g. "denim" → "bottoms"), (4) drop compliance_status, (5) try recommend_suppliers for ranked fit. • "Invalid province" → use English (Guangdong) or standard Chinese (广东). Supported: 31 mainland provinces + HK/Macau. • product_type returns 0 → the TYPO_MAP normalizes common variants; try synonyms ("tee" → "t-shirt", "jeans" → "denim", "运动服" → "activewear"). • Rate limit 429 → wait 60 seconds. Do not retry immediately. • Empty after 3 retries → tell user: "I couldn't find suppliers matching [criteria]. Would you like me to broaden the search?" AVOID: Do not call this tool in a loop across provinces — call get_province_distribution first to see where supply is concentrated. Do not use this for ranked "best fit" recommendations — use recommend_suppliers. Do not fetch details by looping — use compare_suppliers with up to 10 IDs. NOTE: Use this for FILTERING by exact criteria. For ranked recommendations based on sourcing needs, use recommend_suppliers instead. Source: MRC Data (meacheal.ai). 中文：搜索经过核查的中国服装供应商档案，按地区、类型、产能、品类、合规状态等筛选。

Selection evidence

Confusable tool pairs.

9 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.

Tool A	Tool B	Confidence	Why they collide
`get_supplier_fabrics`	`get_fabric_suppliers`	high	Names are exact mirror images ('supplier's fabrics' vs 'fabric's suppliers') and both take a single ID argument (supplier_id vs fabric_id) but return semantically inverted relationships; a request like 'what's available for fab_XXX from this supplier' or 'who sells this fabric' is easy to route to the wrong one.
`search_suppliers`	`recommend_suppliers`	medium	Both are supplier-discovery entry points from natural-language sourcing requests (e.g. 'find/recommend the best factory for X'); the tools' own descriptions explicitly call out the overlap, showing the maintainers anticipated agents confusing filter-based search with ranked recommendation.
`get_supplier_detail`	`get_supplier_fabrics`	medium	Shared 'get_supplier_' prefix and identical supplier_id input; a phrase like 'tell me what this supplier offers' or 'show me sup_XXX's catalog' could plausibly be routed to the full-profile tool instead of the fabric-catalog-specific one.
`get_fabric_suppliers`	`get_cluster_suppliers`	medium	Both list 'suppliers' filtered by a different parent entity (fabric_id vs cluster_id); an ambiguous ask like 'who supplies X in this market/cluster' could lead an agent to pick the wrong ID-scoped suppliers tool.
`recommend_suppliers`	`compare_suppliers`	medium	A vague ask like 'which supplier is best for X' could be interpreted as a ranking request (recommend_suppliers, from requirements) or a comparison request (compare_suppliers, requires existing IDs), especially if the agent already has some candidate IDs in context.
`get_supplier_detail`	`get_fabric_detail`	low	Both follow an identical 'get_X_detail' pattern for a single ID, so a generic 'give me the full details' follow-up without clear entity context could be misrouted, though the required ID prefix (sup_ vs FAB-) usually disambiguates quickly.
`compare_clusters`	`compare_suppliers`	low	Same 'compare_X side-by-side' pattern and phrasing ('compare these', 'benchmark'), but cluster names/IDs vs supplier IDs are usually distinguishable enough that confusion is unlikely except in vague follow-ups referencing 'them'.
`search_suppliers`	`search_clusters`	low	Both are 'search_X' discovery entry points triggered by geographic/product phrasing (e.g. 'where can I find X in Guangdong'), but one returns individual factories and the other returns industrial zones, which is usually clear from whether the user names a product/company vs a region/market.
`get_product_categories`	`get_province_distribution`	low	Both are standalone exploratory overview tools that could be reached for a vague 'what do you have' style question, though one is category-oriented and the other geography-oriented, which is normally clear from the question's focus.

Compare the field

One score is useful.
The evidence makes it actionable.

Back to the leaderboard