Backtesting without self-deception

Write the rule before examining the answer

A backtest applies specified decision rules to historical data and calculates what a modeled sequence of trades would have produced. Its value lies in exposing a hypothesis to evidence. If the rule changes whenever a losing example appears, the exercise becomes a way of describing the past rather than testing a repeatable decision process.

Start with the instrument universe, observation interval, entry condition, exit condition, sizing method and cost model. Define when information becomes available and when an order could first execute. A strategy based on a closing price cannot normally assume that it knew that final close beforehand and filled at the exact same price without further justification.

Watch for information leaking backward

Look-ahead bias occurs when the simulation uses information that was unavailable at the decision time. Examples include revised economic data, a later earnings announcement or a future index membership list. A historical database can be perfectly accurate today yet unsuitable for reconstructing what a person actually knew on an earlier date.

Survivorship bias arises when the tested universe excludes instruments that disappeared. A strategy tested only on companies that still exist may miss bankruptcies, delistings and acquisitions that affected the historical opportunity set. The same issue can arise with failed crypto tokens or discontinued funds. Document how the dataset handles disappearances instead of silently deleting difficult records.

Corporate actions deserve their own audit. In a two-for-one share split, 100 shares at $40 become 200 shares at about $20 before other price changes; the split itself does not halve the position’s value. Mixing adjusted prices with unadjusted share quantities can invent a gain or loss. Dividends also need consistent treatment: do not add a cash dividend to a return series that already includes its reinvestment. Keep the signal-price convention, tradable-price convention and position accounting explicit.

Make execution a model, not a favor

A candle touching a limit price does not prove your entire order would have filled. Queue position, available quantity and the path within the candle matter. If both a stop and a target are touched in one bar, the bar alone may not reveal which happened first. Treat such ambiguity conservatively or use data that can resolve the sequence.

Include spread, commissions, financing, borrowing costs where relevant, and plausible slippage. Tax treatment depends on jurisdiction and circumstances, so a generic simulation should state whether results are pre-tax rather than invent a universal rate. Compare results under several cost assumptions. A method that succeeds only with free and instantaneous execution is fragile.

Separate discovery from evaluation

Trying hundreds of parameter combinations and presenting the best one creates a selection problem. Some combination can look excellent by chance. Reserve data that was not used to develop the rule, and avoid repeatedly consulting that reserve until it becomes another training sample. A forward paper test offers a different check because new observations arrive after the rules are fixed.

Report failed variants, sample size, date range and the number of experiments, not just the winning chart. Examine whether nearby parameter choices behave similarly or whether one isolated setting produces all the profit. Hypothetical results remain hypothetical even when the spreadsheet is precise to several decimal places. Liquidity, emotions and operational problems are not fully reproduced by historical arithmetic.

Keep a research log of data versions and calculation changes. A corrected split adjustment or a newly included delisting can materially alter a result, and that correction should remain visible. Reproducibility means another person can reconstruct the same answer from the stated inputs and rules. It does not mean the answer is economically useful, but it is a necessary step toward finding out.

Reconstruct one trade before trusting a thousand

Pick one representative trade and write its timeline as if you were reconstructing an operational incident. Record the data timestamp, when that data actually became available, the decision timestamp, order arrival, fills, fees and final position. These times are not interchangeable. A filing dated Tuesday but published after the market closes should not influence a Tuesday-morning trade. Time-zone conversions and daylight-saving changes can quietly turn a sensible rule into a look-ahead strategy.

Now reconcile the account, not just the trade list. Starting cash plus cash flows and realized profit or loss, together with the marked value of open holdings, must explain ending equity. A report that counts only closed winners can look healthy while large unrealized losses remain offstage. Likewise, several simultaneous signals cannot each spend the same available cash. The simulator needs a rule for competing orders, insufficient buying power and positions that remain open at the end of the sample.

Execution errors can be tested with tiny artificial datasets whose answers are known. A constant-price series with no fees should not create trading profit. A pure split should leave wealth unchanged after quantities are adjusted. Increasing a per-trade fee, while holding trades fixed, should reduce net profit by exactly the additional fees. A deliberately missing price should produce a documented handling decision, not a silent fill at zero or an indefinitely carried stale quote.

Finally, compare the strategy with a benchmark that answers the actual question. A long-only equity rule may profit simply because the market rose, so compare it with a clearly specified passive exposure over the same dates and with consistent treatment of dividends and cash. A strategy that spends half its time out of the market also changes its exposure and opportunity cost. A better-looking final balance is evidence to investigate, not proof that timing skill caused the difference.