01Safety
30.0 / 30
What changed in the harness
Selection accuracy 100→100, token cost up 2%, unconfirmed writes 100%→100%.
Category breakdown
Where the score comes from.
Earned points across the four signals Gradable measures. Safety and Legibility are scored out of 30; Economics and Discoverability are scored out of 20.
02Legibility
25.8 / 30
03Economics
17.4 / 20
04Discoverability
16.3 / 20
Highest-impact fix
Estimated gain +1 pointRewrite 8 underspecified tool descriptions
State each tool's behavior, explain every parameter, and declare return semantics, prioritizing tools exercised by failed selection tasks.
Description evidence
Defects and rewrites.
8 defects found across the exposed tool descriptions. Suggested rewrites make purpose, inputs, boundaries, and returns easier for an agent to understand.
| Tool | Defect types | Suggested rewrite |
|---|---|---|
opa_get_product_facts |
no_return_description |
Get the reviewed, versioned OilPriceAPI product contract. Returns the authoritative product information covering: product scope, evaluation offer, pricing URL, freshness policy, catalog and entitlement wording, authentication, canonical first request, keyless demo, and data-rights boundaries. Use this instead of model memory or package prose for questions about OilPriceAPI itself. No API key or paid-data entitlement is required. |
opa_get_futures |
no_return_description |
Get the latest front-month futures contract price for energy commodities. Use when the user asks about futures, forward prices, or contract prices. Returns the latest front-month price for the selected contract, identifying the resolved contract (BZ/ice-brent = Brent, CL/ice-wti = WTI, ice-gasoil = ICE Gasoil, natural-gas = Natural Gas, ttf-gas = European TTF gas, lng-jkm = LNG JKM, eua-carbon = EUA carbon, uk-carbon = UK carbon; default BZ). For the full forward curve across all contract months, use opa_get_futures_curve instead. |
opa_get_drilling |
no_return_description |
Get a drilling activity snapshot. Use when the user asks about drilling activity, rigs vs frac spreads, or upstream operations. Returns: US, Canada, and international rig counts, frac spread count, well permits issued in the last 30 days (with a by-state breakdown), and DUC (drilled-uncompleted) well totals. Requires a paid plan with energy intelligence access. |
opa_get_fuel_surcharge |
no_return_description |
Get carrier-published fuel surcharge percentages for LTL freight and parcel carriers. Use when the user asks about current or historical fuel surcharge rates for carriers like ODFL, Saia, Estes, XPO, ABF, TForce, Averitt, Southeastern Freight, UPS, FedEx, or DHL. Returns the latest surcharge percentage for the matched carrier and mode; when history is true, returns up to per_page historical rows (default 12) instead. Auto mode treats UPS/FedEx/DHL as parcel carriers and other carrier slugs as LTL. Parcel history requires a service_level such as ground, air, or international_air_export. |
opa_get_spread |
no_return_description |
Get refining and trading spreads. Use when the user asks about crack spreads, 3-2-1 crack, refining margins, basis differentials, or blend/transport margins. Returns the current values for the selected spread type: crack (refining margin proxy), basis (regional price differentials), or margin (blending/transport margins). Requires a paid plan with energy intelligence access. |
opa_list_price_alerts |
no_return_description |
List all PERSISTENT price alerts on the user's OilPriceAPI account. Use when the user asks what alerts they have set up, or to find an alert's id before deleting it. Returns the account's price alerts with each alert's id and trigger details. REQUIRES an API key (OILPRICEAPI_KEY) — alerts are account-scoped. No parameters needed. |
opa_get_market_brief |
no_return_description |
Get a multi-commodity market brief for several commodities in ONE call. Use when the user wants a market snapshot, morning brief, or an at-a-glance read across multiple commodities. Returns latest spot prices, 24h changes, 1-month forecasts (for Brent/WTI/Natural Gas), and notable spreads for the requested codes. Set narrative: true to also get a plain-English summary plus market context (active supply disruptions, key economic indicators). Accepts natural language ('brent', 'us gas') or API codes. REQUIRES an API key (OILPRICEAPI_KEY); counts as 1 request. Per-tier code limits apply (free: 3 codes). For a single price use opa_get_price; for ongoing recurring monitoring use opa_create_price_subscription. |
opa_list_subscriptions |
no_return_description |
List all PERSISTENT price subscriptions ('watches') on the user's OilPriceAPI account. Use when the user asks what they're monitoring, or to find a watch's id before deleting it. Each watch is a recurring, account-tied snapshot job. Returns the account's watches with each watch's id and the commodity codes it monitors. REQUIRES an API key (OILPRICEAPI_KEY). No parameters needed. |
Selection evidence
Confusable tool pairs.
10 pairs where similar names or overlapping descriptions may send an agent toward the wrong tool.
| Tool A | Tool B | Confidence | Why they collide |
|---|---|---|---|
opa_market_overview |
opa_get_market_brief |
high | Both descriptions explicitly claim to serve 'a market snapshot' across multiple commodities in one call. A task like 'give me a broad market snapshot' or 'market brief' maps to both, especially since overview covers category groups while brief covers codes+forecasts; only the details reveal which is meant. |
opa_get_futures |
opa_get_futures_curve |
medium | A task like 'what are the futures prices for Brent' is ambiguous between the front-month contract (get_futures) and the full forward curve across contract months (get_futures_curve); the user must say 'front-month' or 'curve' to disambiguate. |
opa_get_futures |
opa_get_forecasts |
medium | 'What's the future price of oil/Brent?' is genuinely ambiguous: get_futures returns traded front-month futures contracts, while get_forecasts returns EIA STEO price predictions/outlook. Description keywords ('future', 'price') overlap and only 'prediction' vs 'contract' separates them. |
opa_get_opec_production |
opa_get_well_production |
medium | A task like 'show oil production data' could plausibly select either: OPEC output (country-level) vs US well/state production, since both are simply 'production' data under the same paid energy-intelligence plan; user must specify OPEC or well/US to disambiguate. |
opa_list_price_alerts |
opa_list_subscriptions |
medium | Both are zero-param, API-key account-scoped lists of persistent monitors (price alerts vs recurring 'watches'). A task like 'list what I'm currently monitoring' or even 'show my active price alerts/watches' can map to either, since the user may not distinguish alerts from subscriptions. |
opa_list_subscriptions |
opa_get_subscription_events |
medium | Both concern the user's subscriptions/watches: list returns the active watches, get_subscription_events polls their recorded snapshots. A task like 'show me subscription activity/what changed in my subscriptions' could select either; only the word 'events/catch up' vs 'list' separates them. |
opa_get_alert_triggers |
opa_get_subscription_events |
low | Both fetch recent account activity (fired alerts vs watch snapshots). A task like 'show recent alert/trigger activity' is mostly disambiguated by descriptions, but the overlap of 'alerts firing' and 'events' leaves mild residual ambiguity. |
opa_get_price |
opa_compare_prices |
low | Descriptions are clear (single price vs 2-5 side-by-side), but a task phrased as 'get the price of Brent and WTI' (two commodities) could lead an agent to opa_get_price instead of opa_compare_prices, which explicitly handles 2+. |
opa_get_well_permits |
opa_get_well_production |
low | Both are US 'well' data under the paid energy-intelligence plan, so a vague task like 'latest well data' is ambiguous; descriptions (permitting activity vs production volumes) disambiguate well, but shared prefix and 'data' tokens leave residual risk. |
opa_get_rig_counts |
opa_get_well_permits |
low | Both cover drilling/upstream activity for US oil & gas and share tokens (data, latest, oil, gas, US). A task like 'latest drilling-related data' could plausibly pick either, though descriptions (rig counts vs permits) differ enough to usually disambiguate. |
Compare the field