BEQI: Open-Source Forex Broker Execution Audit Toolkit 04/05/2026 – Posted in: Arbitrage Software

Why a public broker audit didn't exist before

BEQI fixes one specific problem: there's no public methodology, no shared toolkit, and no benchmark dataset for retail forex execution quality. We've been measuring this internally at BJF Trading Group for years to build broker dialect support into SharpTrader Pro and SharpTrader Optimizer. Now we're open-sourcing the methodology and the code.

The point is not to publish a leaderboard naming brokers (that's a legal minefield and would compromise the integrity of the data). The point is to give every trader, quant, and broker-tech researcher the same standardized way to measure, the same Python scripts to run, and a public place to deposit anonymized results. Over months, the spreadsheet becomes a benchmark. Anyone can replicate the methodology, dispute it, or extend it.

The five BEQI dimensions

Execution quality is not a single number. BEQI decomposes it into five orthogonal dimensions, each independently measurable from a trading statement (and, for two of them, from a tick stream paired with the statement):

Each dimension is independently meaningful. A broker can have low matching latency but high slippage asymmetry, or vice versa — the combinations tell different stories about how the broker is actually behaving. The toolkit reports each dimension separately and a composite BEQI score (weighted) at the end.

How to run the audit on your own broker (30 minutes)

You need three things: (a) a trading statement with at least 200 fills covering at least one trading week, (b) Python 3.10+ on any machine, (c) 30 minutes. The toolkit handles the rest.

Step 1 — Export your statement

The toolkit accepts three input formats:

• Common terminal HTML statement (saved from your retail trading platform's “save as report” function) — most retail traders will use this
• FIX log (FIX 4.4 message dump) — for traders running through a FIX bridge or institutional connection
• CSV with columns order_id, send_time, fill_time, requested_price, filled_price, side, volume — for everyone else, including custom logs

If your platform doesn't expose send-time (the moment your strategy decided to send the order), you can still measure dimensions 2 and 5 (slippage asymmetry and requote rate) from a basic statement. Dimensions 1, 3, and 4 require a richer log.

Step 2 — Install and run

The toolkit auto-detects the input format. beqi.py dispatches to the per-dimension analyzers, prints a summary, and writes a JSON report you can submit to the public spreadsheet.

Step 3 — Review the report

The report includes the five dimension scores, the composite BEQI, sample-size diagnostics (lower N = lower confidence), and a tier classification (1 through 4, see below). Run it on more data to tighten the confidence intervals.

Reading your results: the four BEQI tiers

Based on internal measurements across the brokers SharpTrader Pro supports, here are the empirical tiers. These are starting baselines — the public spreadsheet will refine them over time.

Tier classification matters because the same trading strategy will produce wildly different live results across tiers. A latency arbitrage strategy that's profitable on Tier 1 collapses on Tier 3. A trend-following strategy that holds positions for hours is roughly indifferent between Tier 1 and Tier 3 but suffers visibly on Tier 4. Knowing your broker's tier before you commit capital is the entire game.

The public BEQI spreadsheet

The spreadsheet is the long-term asset. Anyone can contribute one row per audit. Each row contains the five dimension scores, the composite, the sample size, the currency pairs covered, the time period, and (optionally, voluntarily) the broker name. We expect — and welcome — anonymous submissions where the broker is left blank; over time, even those produce useful aggregate distributions of execution quality across the retail forex industry.

To submit: run the toolkit, copy the JSON output, paste it into the submission form (link inside the sheet). New rows are reviewed for plausibility and accepted within 24 hours. We do not edit submitted scores; we may flag rows where sample size is <100 fills as low-confidence.

Methodology details (for reviewers and replicators)

The full mathematical specification is in METHODOLOGY.md in the repo, but the load-bearing decisions are:

1. Matching latency

Median (not mean) of fill_time − send_time. We use median because matching latency distributions are right-skewed by occasional 1–5 second outliers (network re-routing, broker GC pauses) that don't represent typical execution. We report median, p90, and p99 separately so the reader can see the tail.

2. Slippage asymmetry

For each fill, slippage is signed: positive means the trader got a better price than requested, negative means worse. Asymmetry = (count of negative slippage events) / (count of positive slippage events). A symmetric, neutral broker scores ~1.0. Above 1.2 indicates the broker's matching is biased against the trader — either by routing rules, by selective last-look, or by an asymmetric slippage engine.

3. Spread widening factor

For each order timestamp, we measure spread in a 50 ms window centered on the order vs spread in the 5-minute baseline window centered on the same hour-of-day across the prior week. The ratio is the widening factor. This requires a tick stream covering the audit period; the toolkit accepts BJF Feed CSVs and the standard tick-history format from common platforms.

4. Last-look hold time

Only measurable from a FIX log or an extended statement that exposes both the matched-but-pending state and the confirmed state. Standard retail HTML statements typically don't expose this, in which case the toolkit reports null for this dimension and the composite BEQI is computed from the remaining four. Last-look is one of several broker-side execution interference techniques — see our deep-dive on the seven anti-arbitrage broker plugins for the broader category.

5. Requote rate

Counted from explicit requote events in the statement. If the statement doesn't flag requotes (some terminals don't), the toolkit infers them by detecting send/cancel/resend triplets within a 200 ms window.

Composite BEQI

Each dimension is mapped to a 0–100 sub-score using the tier definitions above. The composite is a weighted geometric mean: BEQI = (L^0.30 × A^0.25 × W^0.20 × H^0.15 × R^0.10) where L, A, W, H, R are the five sub-scores. We use geometric mean rather than arithmetic so that a catastrophic score on any single dimension (e.g., last-look = 1500ms) cannot be hidden by good scores on others. Weights are debatable and explicitly published so reviewers can recompute under different weights from the raw sub-scores.

Limitations and disclosures

• Sample size dependency. A 50-fill audit is a vibe; a 500-fill audit is a measurement; a 5,000-fill audit is a benchmark. The toolkit reports sample-size confidence intervals; readers should weight low-N submissions accordingly.
• Time-of-day variance. The same broker can post Tier 2 results during London open and Tier 3 during Asia rollover. Audits should specify the time window.
• Account-level differences. Brokers can apply different routing logic per-account based on profile, balance, and trading pattern. One trader's audit may not generalize to another's on the same nominal broker. For broker-selection guidance independent of execution audit, see our forex arbitrage brokers guide.
• Asymmetry doesn't prove malice. An asymmetry of 1.4 is statistically loud, but the cause might be routing through asymmetric LP pools rather than active manipulation. The toolkit measures behavior, not intent.
• The methodology will be wrong about something. That's why it's open-source. Submit issues, send PRs, propose alternative weights, replicate against a broker we haven't covered. Methodology v2 will incorporate the strongest critiques.

Roadmap

Frequently asked questions