What Is the Bias-Variance Tradeoff? A Plain-Language Explanation

Updated · By Viviai

The bias-variance tradeoff is the tension in machine learning where reducing a model's bias by making it more flexible tends to increase its variance, and the goal is to balance both to keep generalization error low.

The simple analogy

A rigid thinker may miss nuance, while an overly impressionable thinker changes their mind with every new story. Bias is the error from being too rigid; variance is the error from being too sensitive to the data at hand.

The two errors

Source Meaning Associated with
Bias Error from oversimplified assumptions Underfitting
Variance Sensitivity to changes in training data Overfitting

How complexity moves them

As a model becomes more complex, bias generally falls while variance rises. A model that is too simple shows high bias; one that is too flexible shows high variance.

The irreducible part

Expected error also includes noise inherent to the data, which cannot be removed by the model.

Honest caveats

The classical tradeoff is a useful framing, although modern overparameterized models have led researchers to revisit how the relationship behaves in practice.

FAQ

What is bias?

Error caused by a model being too simple to capture the underlying pattern.

What is variance?

How much the model's predictions change when trained on different data.

How do bias and variance relate to fitting?

High bias links to underfitting, high variance links to overfitting.

Can both be eliminated?

They can be reduced, but data noise sets a floor on achievable error.

Key takeaways

Continue with what is regularization and what is overfitting.

Sources

  1. https://www.ibm.com/think/topics/bias-variance-tradeoff
  2. http://www.shturl.cc/eb441e97221580740ff449e8392ce83e
  3. https://cs.uwaterloo.ca/~jhoey/teaching/cs486/biasvariance.pdf

Learn by doing

Read a guide, then practice with bite-sized AI lessons you can take offline.

Start learning free