What Is Hierarchical Clustering? Agglomerative Method and Dendrograms Explained
Hierarchical clustering is an unsupervised method that groups data into a nested sequence of clusters and displays them as a tree called a dendrogram. Instead of choosing a fixed number of groups up front, you build the full hierarchy and then "cut" the tree at the level that gives the number of clusters you want. The more common version, agglomerative clustering, starts with every data point in its own cluster and repeatedly merges the two closest groups until one cluster remains.
The Short Answer: How It Works
Hierarchical clustering answers the same question as other clustering methods — which points belong together — but keeps the relationships between groups rather than only a flat partition.
There are two directions:
- Agglomerative (bottom-up): begin with one cluster per point, then merge the nearest pair at each step.
- Divisive (top-down): begin with all points in one cluster, then split it recursively.
Agglomerative clustering is used more often because its steps are simple to implement and explain. The rest of this article focuses on that approach.
Agglomerative Clustering, Step by Step
The algorithm is a repeating loop driven by a distance measure and a linkage rule.
- Treat each of the (n) data points as its own singleton cluster.
- Compute a matrix of pairwise distances between every cluster.
- Find the two clusters that are closest under the chosen linkage.
- Merge them into a new cluster and record the merge height.
- Update the distance matrix to reflect the new cluster.
- Repeat from step 3 until a single cluster containing all points remains.
The recorded merges form the dendrogram. Each merge is a greedy local decision: once two groups are joined, they are not separated again.
Reading a Dendrogram
A dendrogram is an upside-down tree that encodes the entire merge history.
- Leaves (bottom): individual observations, each starting as its own cluster.
- Internal nodes: merge events where two groups joined.
- Root (top): the single cluster holding every point.
- Height of a node: the dissimilarity at which its two branches merged.
The vertical axis is the meaningful one. The horizontal order of leaves is largely arbitrary and does not by itself indicate similarity.
Cutting the Tree to Choose K
To obtain a flat set of clusters, you draw a horizontal line across the dendrogram at a chosen height. Every branch below that line becomes one cluster. A lower cut produces more, tighter clusters; a higher cut produces fewer, broader ones. This is how you can explore several values of (K) without rerunning the algorithm.
A useful property follows from the construction: clusters are nested. Every cluster formed at a lower height is a subset of a cluster formed at a higher height. That nesting cannot be violated.
Linkage Methods: How Distance Between Clusters Is Defined
"Distance between two points" is not the same as "distance between two groups of points." The linkage criterion defines the latter and changes the shape of the result.
| Linkage | Distance between two clusters | Tendency |
|---|---|---|
| Single linkage | Smallest distance between any pair of points | Can form long, chained clusters |
| Complete linkage | Largest distance between any pair of points | Favors compact, similarly sized groups |
| Average linkage | Mean of all pairwise distances | A middle ground between the two |
| Ward's method | Increase in total within-cluster variance after merging | Produces balanced, variance-minimizing groups |
| Centroid linkage | Distance between cluster centroids | Intuitive, but merges can be non-monotonic |
Single linkage can chain together points that are only loosely connected, while complete and Ward linkage tend to produce more compact clusters. The choice of linkage is often as influential as the choice of distance metric, which is commonly Euclidean distance for continuous data.
Hierarchical Clustering vs K-Means vs DBSCAN
Each approach makes different assumptions about the data.
| Feature | Hierarchical | K-Means | DBSCAN |
|---|---|---|---|
| Set number of clusters in advance | No (cut later) | Yes, choose (K) | No |
| Output shape | Nested tree, dendrogram | Flat partition | Flat partition plus noise label |
| Cluster shapes it handles | Depends on linkage | Mostly round, equal-scale groups | Arbitrary shapes via density |
| Handles outliers | Sensitive, depending on linkage | Sensitive | Labels outliers as noise |
| Typical cost on (n) points | Often around (O(n^3)) time, (O(n^2)) memory | Lower, scales to large data | Around (O(n \log n)) with indexing |
| Interpretability | High, shows group relationships | Moderate | Moderate |
For more on the alternatives, see the overviews of clustering, K-Means, and DBSCAN, all within the broader topic of unsupervised learning.
Strengths and Limitations
Strengths
- You do not commit to a single (K) up front; the dendrogram lets you inspect several cuts.
- Any valid distance measure can be used, and only the distance matrix is required — the raw observations are not strictly needed.
- The tree is visual and interpretable, which helps explain group structure to non-technical stakeholders.
Limitations
- The standard agglomerative algorithm is computationally heavy, commonly cited near (O(n^3)) in time and (O(n^2)) in memory, which makes the plain version impractical for very large datasets without sampling or approximations.
- Merges are irreversible. A poor early decision propagates upward and cannot be undone.
- Results are sensitive to the distance metric, linkage, and feature scaling; variables on larger scales can dominate if features are not normalized.
Common Use Cases
- Taxonomy and biology: organizing species or genes into hierarchical families, the historical origin of the method.
- Document and topic grouping: building nested groups of articles, support tickets, or search results.
- Customer and product segmentation: exploring customer groups at different levels of granularity from one tree.
- Social science and survey research: grouping respondents when relationships across levels matter.
Frequently Asked Questions
Do I need to decide the number of clusters first?
No. Agglomerative clustering builds the complete hierarchy, and you choose the number of groups afterward by cutting the dendrogram at a height that fits the data. This differs from K-Means, where (K) is set before training.
What is the difference between agglomerative and divisive clustering?
Agglomerative clustering starts with every point separate and merges groups upward. Divisive clustering starts with one cluster and splits it downward. Both produce a hierarchy, but the agglomerative direction is more widely implemented.
How do I choose a linkage method?
A practical starting point is Ward's method or average linkage for compact, balanced groups, with Euclidean distance on scaled features. Single linkage is useful when you specifically want to detect connected chains but is prone to merging clusters through a few close points. Testing more than one linkage and comparing dendrograms is a reasonable approach.
When should I use hierarchical clustering instead of K-Means?
Consider it when you want a visual, nested view of relationships, prefer not to fix (K) in advance, or only have pairwise distances. For very large datasets where speed matters, K-Means or density-based methods are usually easier to scale.
Summary
Hierarchical clustering builds a nested tree of groups, and agglomerative clustering does so by repeatedly merging the closest clusters until one remains. The dendrogram records every merge and lets you cut to any number of clusters afterward. Its main trade-off is interpretability and flexibility against higher computational cost and irreversible greedy merges, which makes it a strong fit for small to medium datasets where understanding group structure matters.
Sources
- https://www.cs.princeton.edu/courses/archive/fall18/cos324/files/hierarchical-clustering.pdf
- https://www2.stat.duke.edu/~rcs46/lectures_2015/05-clus2/05-clus2.pdf
- https://cse.buffalo.edu/~jing/cse601/fa13/materials/clustering_hierarchical.pdf
- https://en.wikipedia.org/wiki/Hierarchical_clustering
- https://scikit-learn.org/stable/modules/clustering.html#hierarchical-clustering
Learn by doing
Read a guide, then practice with bite-sized AI lessons you can take offline.
Start learning free