What Is Survivorship Bias in Backtesting? The Hidden Reason Results Look Too Good

“I backtested my strategy against the TOPIX constituents over the last 10 years and got +120% return” — numbers like these make a strategy feel validated.

But there’s an easy-to-miss premise: today’s stock list doesn’t include companies that were delisted during those 10 years. This is survivorship bias, and when left unchecked, it makes your backtest results consistently more optimistic than reality.

What Is Survivorship Bias?

Survivorship bias is the statistical distortion that emerges when you only observe “what survived,” leading you to overestimate the overall trend. It’s been discussed in investing, statistics, and history for decades.

The classic example is statistician Abraham Wald analyzing WWII bomber bullet holes. Examining only the damage patterns on returned planes ignores the pattern on planes that were shot down — leading you to reinforce the wrong parts.

How It Shows Up in Backtests

If you only backtest against currently-listed instruments, you miss the impact of companies that disappeared through bankruptcy, delisting, or acquisition:

• Bankrupt companies: their price drops to zero is excluded
• Poor-performance delistings: the large pre-delisting drops are excluded
• Major acquisitions: buyout premiums are excluded (positive-direction bias)

In general, currently-listed instruments are “the ones not culled by past markets,” so results tend to be structurally better than actual historical reality.

How Large Is the Impact?

The magnitude depends on the universe (set of instruments being tested) and the period.

Universe	Impact size
Large-cap like TOPIX Core30	Relatively small (rarely culled)
Small/mid-cap in Mothers or Growth markets	Large (many delistings)
Long periods like 20+ years	Larger
Short periods like 1–2 years	Smaller

Especially when testing a universe with many small-caps over a long period, survivorship bias is not negligible.

Fully Avoiding It Is Hard

Fully eliminating survivorship bias requires point-in-time historical universes that include delisted instruments. This data is usually sold by commercial data providers and isn’t readily available to individual investors.

QuanTest, like most consumer backtesting tools, uses currently-listed instruments as its base, so it’s not completely free from this bias. For background on the data availability tradeoffs, see Why QuanTest chose CSV import.

Practical Mitigations

Since you can’t fully avoid it, the realistic approach is to correct for it when interpreting results:

• Treat reported returns as containing a survivorship-bias uplift — discount them
• Test against large-cap-heavy universes to keep the impact relatively small
• Be especially cautious with results from periods with many delistings (e.g., around the 2008 financial crisis)

Adopting a conservative “assume the real number is lower than what I see” stance closes the gap between backtested and live performance. Combine this with how to read each metric for a complete evaluation framework.

The Value of Doubting Your Results

Rather than taking backtested numbers at face value, get into the habit of asking “is this more optimistic than reality?” Survivorship bias is often the first concept that triggers that habit. Pair it with overfitting for a solid mental toolkit against misleading results.

Try It in QuanTest

With QuanTest, you can switch between universes and periods and see how much results change. Whether you have the material to doubt a flashy number when you see one is what separates short-term luck from long-term validation skill.

---

This article is for educational purposes. It does not guarantee the profitability of any strategy or future performance. Investment decisions are your own responsibility.