What Is the Bias-Variance Tradeoff? A Plain-Language Explanation
The bias-variance tradeoff is the tension in machine learning where reducing a model's bias by making it more flexible tends to increase its variance, and the goal is to balance both to keep generalization error low.
The simple analogy
A rigid thinker may miss nuance, while an overly impressionable thinker changes their mind with every new story. Bias is the error from being too rigid; variance is the error from being too sensitive to the data at hand.
The two errors
| Source | Meaning | Associated with |
|---|---|---|
| Bias | Error from oversimplified assumptions | Underfitting |
| Variance | Sensitivity to changes in training data | Overfitting |
How complexity moves them
As a model becomes more complex, bias generally falls while variance rises. A model that is too simple shows high bias; one that is too flexible shows high variance.
The irreducible part
Expected error also includes noise inherent to the data, which cannot be removed by the model.
Honest caveats
The classical tradeoff is a useful framing, although modern overparameterized models have led researchers to revisit how the relationship behaves in practice.
FAQ
What is bias?
Error caused by a model being too simple to capture the underlying pattern.
What is variance?
How much the model's predictions change when trained on different data.
How do bias and variance relate to fitting?
High bias links to underfitting, high variance links to overfitting.
Can both be eliminated?
They can be reduced, but data noise sets a floor on achievable error.
Key takeaways
- Bias and variance move in opposite directions with complexity.
- Bias signals underfitting, variance signals overfitting.
- Balancing them supports generalization.
- Some error from noise is irreducible.
Continue with what is regularization and what is overfitting.
Sources
Learn by doing
Read a guide, then practice with bite-sized AI lessons you can take offline.
Start learning free