Backtesting

If you’ve ever watched a trading tutorial, you’ve met the opening-range breakout: mark the high and low of the first minutes of the session, and trade the breakout when price escapes that range. It’s simple, mechanical, and everywhere — and recent academic papers report spectacular returns for it on stocks. We wanted to know: does it actually work on futures — the markets we research — once you pay real-world trading costs? How we made it impossible to fool ourselves # The problem with backtests is that the person running them controls everything: which variants get tried, which costs get assumed, which results get shown. Try enough variants and something will always look great by accident. So before computing a single profit number, we pre-registered the entire experiment in our public repository: nine futures markets (Nasdaq, S&P, Russell, Dow, gold, silver, copper, crude oil, natural gas), two session anchors, four range lengths (5/15/30/60 minutes), three exit rules taken verbatim from the literature, plus a published comparator rule — 225 configurations, zero tunable parameters, verdict thresholds and controls fixed in advance. Sixteen years of one-minute data (2010–2026). One contract per trade. Costs stressed at $25 per round trip. No configuration could be added, removed, or “fixed” after seeing results.

Everyone forward-tests other people’s strategies. The uncomfortable experiment is doing it to your own — the 54 optimizer-selected configurations you actually believe in — on a window of the future that arrived after every parameter was fixed, with a falsifier written before the run. We did it twice (round two after a data correction — more on that below). Here’s what honest out-of-sample performance looks like. The headline table # 3,733 entirely new trades across all 54 configurations, nine futures markets: Fresh window, all 54 trades raw at $10/round-trip at $25/round-trip total 3,733 +$29,807 −$7,523 −$63,518 The raw positive is statistically indistinguishable from zero (t = 0.88). Against what the calibration window promised, the fleet kept 17.6% of its per-trade rate — a decay that is itself statistically significant (t = −2.53). The survivors of $25 costs are few and nameable: the 4-hour timeframe (+$10,106 net) and ES as an instrument (+$17,119). Fifteen of 54 configurations stayed positive at $25; most of the rest are small-timeframe cells whose $4–7 gross per trade is a commission illusion.