4 Checkpoints for Avoiding Overfitting in Backtests

“I tweaked a parameter slightly and got a great number.” Everyone who starts using a backtesting tool experiences this temptation.

But pushed too far, “optimization” just produces a phantom strategy that’s fitted to past data and nothing else. This is overfitting, a classic pitfall in machine learning. The exact same problem shows up in backtesting.

What Is Overfitting?

Overfitting is when you pick a parameter combination that happens to work well on past data by chance. Apply the same parameters to a different period or instrument, and the result doesn’t reproduce.

It’s most commonly discussed in machine learning, but the issue is identical in trading strategy validation.

Telltale Signs

Overfitting is likely when you see:

• Backtest results deteriorate sharply with a small parameter change
• Max drawdown is unnaturally small relative to cumulative return
• Win rate is extremely high (80%+)
• Trade count is extremely low (a few times a year — sample size too small)
• The logic looks like it was designed to hit a specific historical period

In particular, grid search — trying every parameter combination and picking the best performer — is the classic overfitting trap.

Why It Happens

Past data contains both signal (what the strategy’s logic should capture) and random noise. The finer you tune parameters, the more your strategy fits the noise. Noise doesn’t reproduce in the future, so the strategy fails in live trading.

Larger datasets improve the signal-to-noise ratio, but the limited sample size of stock price data means overfitting risk is always present.

How to Reduce the Risk

You can’t eliminate it, but you can reduce it:

1. Split into In-Sample / Out-of-Sample

Divide past data in two. Decide parameters on the first half (in-sample), then check results on the second half (out-of-sample). If the second half performs far worse than the first, that’s an overfitting signal.

2. Verify Stability Across Multiple Instruments and Periods

Check whether parameters that worked on one instrument also work on others to a similar degree. A strategy that only works on cherry-picked instruments has questionable generality.

3. Check Parameter Sensitivity

See whether small shifts around the optimum still produce stable results. If only the optimum looks good and adjacent values collapse, you likely got lucky with a specific combination. For instance, with the moving average cross, compare short 5 vs 6 / long 25 vs 30 side by side to visualize sensitivity.

4. Prefer Simplicity

More parameters = more room for overfitting. Strategies with fewer parameters and simpler logic are structurally less likely to be fitted to specific historical patterns.

Pair with Result Evaluation

When evaluating an optimized strategy, look beyond return and check max drawdown and sensitivity. On top of that, if you’re only testing against currently-listed instruments, survivorship bias inflates your numbers — factor that in when reading results.

A Validation Routine in QuanTest

With QuanTest you can re-run the same strategy across different periods and instruments. Instead of jumping on the first number that appears, make it a habit to confirm results across multiple conditions. That habit is the first step in avoiding overfitting.

“The moment a flashy number shows up is the moment to doubt it.” Try practicing that discipline hands-on.

---

This article is for educational purposes. It does not guarantee the profitability of any strategy or future performance. Investment decisions are your own responsibility.