We forward-tested our own strategies on data they'd never seen. They kept 17.6%

contact@mulhamfetna.com (Mulham Fetna) — Mon, 07 Sep 2026 00:00:00 +0000

Everyone forward-tests other people’s strategies. The uncomfortable experiment is doing it to your own — the 54 optimizer-selected configurations you actually believe in — on a window of the future that arrived after every parameter was fixed, with a falsifier written before the run.

We did it twice (round two after a data correction — more on that below). Here’s what honest out-of-sample performance looks like.

The headline table
#

3,733 entirely new trades across all 54 configurations, nine futures markets:

Fresh window, all 54	trades	raw	at $10/round-trip	at $25/round-trip
total	3,733	+$29,807	−$7,523	−$63,518

The raw positive is statistically indistinguishable from zero (t = 0.88). Against what the calibration window promised, the fleet kept 17.6% of its per-trade rate — a decay that is itself statistically significant (t = −2.53). The survivors of $25 costs are few and nameable: the 4-hour timeframe (+$10,106 net) and ES as an instrument (+$17,119). Fifteen of 54 configurations stayed positive at $25; most of the rest are small-timeframe cells whose $4–7 gross per trade is a commission illusion.

We tested the most famous day-trading strategy on 16 years of data. Zero of 225 versions survived real costs.

contact@mulhamfetna.com (Mulham Fetna) — Mon, 07 Sep 2026 00:00:00 +0000

If you’ve ever watched a trading tutorial, you’ve met the opening-range breakout: mark the high and low of the first minutes of the session, and trade the breakout when price escapes that range. It’s simple, mechanical, and everywhere — and recent academic papers report spectacular returns for it on stocks.

We wanted to know: does it actually work on futures — the markets we research — once you pay real-world trading costs?

How we made it impossible to fool ourselves
#

The problem with backtests is that the person running them controls everything: which variants get tried, which costs get assumed, which results get shown. Try enough variants and something will always look great by accident.

So before computing a single profit number, we pre-registered the entire experiment in our public repository: nine futures markets (Nasdaq, S&P, Russell, Dow, gold, silver, copper, crude oil, natural gas), two session anchors, four range lengths (5/15/30/60 minutes), three exit rules taken verbatim from the literature, plus a published comparator rule — 225 configurations, zero tunable parameters, verdict thresholds and controls fixed in advance. Sixteen years of one-minute data (2010–2026). One contract per trade. Costs stressed at $25 per round trip. No configuration could be added, removed, or “fixed” after seeing results.

Backtesting on Mulham Fetna

We forward-tested our own strategies on data they'd never seen. They kept 17.6%

The headline table #

We tested the most famous day-trading strategy on 16 years of data. Zero of 225 versions survived real costs.

How we made it impossible to fool ourselves #

The headline table
#

How we made it impossible to fool ourselves
#