We forward-tested our own strategies on data they'd never seen. They kept 17.6%
Everyone forward-tests other people’s strategies. The uncomfortable experiment is doing it to your own — the 54 optimizer-selected configurations you actually believe in — on a window of the future that arrived after every parameter was fixed, with a falsifier written before the run.
We did it twice (round two after a data correction — more on that below). Here’s what honest out-of-sample performance looks like.
The headline table # 3,733 entirely new trades across all 54 configurations, nine futures markets:
Fresh window, all 54 trades raw at $10/round-trip at $25/round-trip total 3,733 +$29,807 −$7,523 −$63,518 The raw positive is statistically indistinguishable from zero (t = 0.88). Against what the calibration window promised, the fleet kept 17.6% of its per-trade rate — a decay that is itself statistically significant (t = −2.53). The survivors of $25 costs are few and nameable: the 4-hour timeframe (+$10,106 net) and ES as an instrument (+$17,119). Fifteen of 54 configurations stayed positive at $25; most of the rest are small-timeframe cells whose $4–7 gross per trade is a commission illusion.