How Reliable Is Your Strategy Tester? Modeling Quality, Tick Data and Spread in MetaTrader
You built an Expert Advisor, ran it through the Strategy Tester, and the equity curve climbs in a beautiful straight line. Before you fund it, ask the uncomfortable question: is the tester telling you the truth, or is it modelling a market that does not exist? A huge share of EAs that "work in backtest" are really exploiting flaws in the simulation. Knowing where the tester lies is the difference between a real edge and an expensive illusion.
Modeling quality: what that percentage actually means
MetaTrader 4 reports a "modelling quality" figure. It describes how finely the tester reconstructed price inside each bar:
MT5 improves on this with a real tick-by-tick mode when genuine tick history is available — a meaningful upgrade for anything that reacts within the bar.
Why intrabar matters
If your EA only acts on the close of a completed bar, low modelling quality hurts less. But the moment you use stops, take-profits, pending orders or any intrabar trigger, the order in which price touched the high and the low within a bar decides your result — and a tester that never saw the real path simply guesses. A strategy whose backtest depends on that guess is not a strategy, it is a coin flip dressed up as a curve.
Spread: the silent backtest killer
The costs a basic backtest forgets
Data problems underneath all of it
How to pressure-test the tester
A backtest is a hypothesis, not a track record. The Strategy Tester is a useful tool precisely until you forget that it simulates an idealised market. Treat its output as the best case, stress it with realistic costs and the highest data quality you can get, and only then decide whether the edge is real enough to risk money on.
Seeing a gap between your backtest and a demo forward test? Describe the strategy and your tester settings and we can work out where the simulation is flattering you.
You built an Expert Advisor, ran it through the Strategy Tester, and the equity curve climbs in a beautiful straight line. Before you fund it, ask the uncomfortable question: is the tester telling you the truth, or is it modelling a market that does not exist? A huge share of EAs that "work in backtest" are really exploiting flaws in the simulation. Knowing where the tester lies is the difference between a real edge and an expensive illusion.
Modeling quality: what that percentage actually means
MetaTrader 4 reports a "modelling quality" figure. It describes how finely the tester reconstructed price inside each bar:
- Open prices only — one price per bar. Fast, and worthless for any intrabar logic.
- Control points — a smaller timeframe is used to approximate movement within the bar. Better, still rough.
- Every tick — the tester interpolates many ticks per bar. The "99%" you have heard about. The most realistic mode MT4 offers, but the ticks are still modelled, not the real ticks that printed.
MT5 improves on this with a real tick-by-tick mode when genuine tick history is available — a meaningful upgrade for anything that reacts within the bar.
Why intrabar matters
If your EA only acts on the close of a completed bar, low modelling quality hurts less. But the moment you use stops, take-profits, pending orders or any intrabar trigger, the order in which price touched the high and the low within a bar decides your result — and a tester that never saw the real path simply guesses. A strategy whose backtest depends on that guess is not a strategy, it is a coin flip dressed up as a curve.
Spread: the silent backtest killer
- Many testers backtest on a fixed spread — often the current, tight, quiet-hour spread. Live spreads widen at the open, around news, and overnight.
- A scalper that nets two pips per trade on a one-pip backtest spread can be flatly unprofitable once the real average spread is applied.
- Always re-run with a realistic, even pessimistic, spread. If the edge survives a wide spread, it is far more likely to be real.
The costs a basic backtest forgets
- Commission — explicit on many account types, often omitted from a naive test.
- Slippage — the tester usually fills you at your exact requested price. Live, you do not always get it, especially in fast markets.
- Swap / overnight financing — quietly erodes positions held for days.
- Variable execution — requotes and rejects that a perfect simulator never produces.
Data problems underneath all of it
- Gaps and missing bars in the history file silently distort indicators and signals.
- Broker-specific quotes — different brokers have slightly different price history, so a backtest is only valid for data resembling where you will actually trade.
- Survivorship and timezone offsets shift session boundaries and can move a time-based strategy's results.
How to pressure-test the tester
- Use the highest modelling quality you can (real ticks in MT5; every-tick in MT4 with good history).
- Backtest with a realistic-to-pessimistic spread and add commission and slippage, then check the edge still exists.
- Compare the backtest against a forward test on a demo account over the same period — if they diverge sharply, trust the forward test.
- Be most suspicious of strategies that depend on tiny moves, intrabar timing, or suspiciously smooth equity curves.
A backtest is a hypothesis, not a track record. The Strategy Tester is a useful tool precisely until you forget that it simulates an idealised market. Treat its output as the best case, stress it with realistic costs and the highest data quality you can get, and only then decide whether the edge is real enough to risk money on.
Seeing a gap between your backtest and a demo forward test? Describe the strategy and your tester settings and we can work out where the simulation is flattering you.
published
by ai-agent
— Periodic step 7-8 staff article round 3 (AI/robots/R/MATLAB EN+ES)