What Is K-Nearest Neighbors (KNN)? A Plain-Language Explanation

Updated · By Viviai

K-nearest neighbors (KNN) is a supervised, instance-based algorithm that classifies a new point by looking at its K closest training examples and taking a majority vote, or averaging their values for regression.

The simple analogy

If you move to a new neighborhood, you can guess what the area is like by looking at the people living nearest to you. KNN does the same: it assumes similar points are located near each other.

How a prediction is made

  1. Measure the distance from the new point to every training point.
  2. Sort the distances and select the K nearest.
  3. For classification, take the most common label; for regression, take the average.

The role of K

Value of K Typical effect
Small K, such as 1 Sensitive to individual noisy points
Larger K Smoother decision, but may overlook local patterns

The number K is usually chosen through validation rather than guessed.

How distance is measured

A common choice is Euclidean distance, the straight-line distance between points. Other metrics, such as Manhattan distance, are used depending on the data.

Where it is commonly used

KNN is used for recommendation, simple classification, anomaly detection, and as the retrieval mechanism behind vector search.

Honest caveats

KNN does little work during training but can be slow at prediction time because it compares against every stored point, and features on different scales can distort the distance, so scaling is often helpful.

FAQ

What does K stand for?

The number of nearest neighbors the algorithm considers when making a prediction.

Is KNN used for classification or regression?

Both, through majority voting or averaging the neighbors' values.

Why is feature scaling important?

Without it, features measured in larger numbers can dominate the distance calculation.

What is a distance metric?

A rule for measuring how far apart two points are, such as Euclidean distance.

Key takeaways

Continue with what is a support vector machine and what is clustering.

Sources

  1. https://www.cs.cornell.edu/courses/cs4758/2012sp/materials/cs4758-knn-lectureslides.pdf
  2. https://www.elastic.co/what-is/knn
  3. https://aiwiki.ai/wiki/k_nearest_neighbors

Learn by doing

Read a guide, then practice with bite-sized AI lessons you can take offline.

Start learning free