Search enough trading rules and one will look brilliant — on data with no edge at all. Pull the lever: it tries hundreds of moving-average strategies, keeps the best-looking one, then checks it out-of-sample. Watch the "alpha" evaporate.
data:
🎲 Pure random walk📈 Market-like
strategies tried: 200
Best in-sample Sharpe
—
Out-of-sample Sharpe
—
Deflated Sharpe
—
in-sample (where we cherry-picked)out-of-sample (the honest test)
—
Every strategy we tried (in-sample Sharpe) — we keep only the red one
Deflated Sharpe vs # strategies — more searching → harder to be real
Educational — and the opposite of advice. The point is that a strategy chosen by searching its own backtest tells you almost nothing about the future. The Deflated Sharpe Ratio (Bailey & López de Prado) prices in how many strategies you tried; below ~0.95 it's indistinguishable from luck. On a pure random walk, no edge can exist — yet the casino still hands you a Sharpe above 1.