AI.Cruz - Learn AI Through Interactive Visualizations

MACHINE LEARNING

Cluster Validation: Internal, External, and Relative Criteria

How do you know if your clusters are good? Internal criteria measure cohesion and separation from the data alone. External criteria compare to ground-truth labels. Relative criteria compare different clustering runs.

1Internal Validation: No Labels Required

Internal indices evaluate clustering quality using only the data and the cluster assignments — no ground-truth labels. They quantify two competing goals: cohesion (points within a cluster should be close to each other) and separation (clusters should be far from each other).

SSW (Within-cluster scatter)
Sum of squared distances from each point to its cluster centroid. Measures cohesion — lower is better. Used in the elbow method to pick K.

SSB (Between-cluster scatter)
Sum of squared distances from each cluster centroid to the global centroid, weighted by cluster size. Measures separation — higher is better.

Calinski-Harabasz index
Ratio SSB/SSW × (n-k)/(k-1). Higher = better-defined clusters. Like an F-ratio for clustering: penalises both poor separation and poor cohesion.

Silhouette coefficient: for each point xᵢ, let a(xᵢ) = mean distance to points in its own cluster, and b(xᵢ) = mean distance to points in the nearest other cluster. Then SC(xᵢ) = (b(xᵢ) - a(xᵢ)) / max(a(xᵢ), b(xᵢ)). Range: [-1, 1]. SC near 1 = point is well-matched to its cluster; near 0 = on a boundary; near -1 = probably in the wrong cluster. The mean SC across all points is the overall silhouette score.

2External Validation: Comparing to Ground Truth

External indices compare the clustering result to a known set of true class labels. They answer: does the algorithm recover the natural groupings? These are useful in research when you have labeled data but want to evaluate clustering without using labels during training.

For any pair of points (i, j) you can classify the pair into four categories based on whether the clustering and the ground truth agree:

a: same cluster AND same true class (true positive)
b: same cluster BUT different true class (false positive)
c: different cluster AND same true class (false negative)
d: different cluster AND different true class (true negative)

Rand Index
RI = (a + d) / (a + b + c + d). Fraction of pairs correctly classified by the clustering. Range [0,1]; 1 = perfect match. Naive baseline ≈ 0.5 for random clusters.

Jaccard Index
J = a / (a + b + c). Ignores true negatives (d) — only looks at pairs that should be together or are grouped together. More informative when most pairs are in different clusters.

3Relative Validation: Comparing Different Runs

Relative criteria compare multiple clustering results to find the best K or the best algorithm setting. No ground truth needed — you just run the algorithm many times and compare the internal index scores.

Elbow method: plot SSW vs. K. As K increases, SSW always decreases. The optimal K is at the "elbow" — where the rate of decrease drops sharply. Adding more clusters beyond the elbow gives diminishing returns in cohesion. Combine with silhouette scores to confirm: the silhouette peaks at the same K.

When to use internal criteria
Any unsupervised problem — you have no labels. Use silhouette + elbow to select K. Plot the silhouette for each K and pick the one with the highest mean score.

When to use external criteria
Benchmarking on labeled datasets, comparing algorithms. Use Adjusted Rand Index (ARI = RI corrected for chance) for fair comparison — standard RI can be inflated by a large number of pairs.

K-Means →DBSCAN →Elbow & Silhouette →

Compute the silhouette coefficient from scratch with numpy

import numpy as np

def euclidean_dist(a, b):
    return float(np.sqrt(np.sum((a - b) ** 2)))

def silhouette_score(X, labels):
    n = len(X)
    scores = []
    unique_labels = np.unique(labels)
    for i in range(n):
        own = labels[i]
        # a(i): mean distance to other points in same cluster
        own_pts = [X[j] for j in range(n) if labels[j] == own and j != i]
        if len(own_pts) == 0:
            scores.append(0.0)
            continue
        a = np.mean([euclidean_dist(X[i], p) for p in own_pts])
        # b(i): min mean distance to any other cluster
        other_means = []
        for lbl in unique_labels:
            if lbl == own:
                continue
            other_pts = [X[j] for j in range(n) if labels[j] == lbl]
            if other_pts:
                other_means.append(np.mean([euclidean_dist(X[i], p) for p in other_pts]))
        b = min(other_means) if other_means else 0.0
        sc = (b - a) / max(a, b) if max(a, b) > 0 else 0.0
        scores.append(sc)
    return scores

# Three 2D clusters
X = np.array([
    [1.0, 1.0], [1.5, 1.2], [0.8, 1.4],   # cluster 0
    [5.0, 5.0], [5.3, 4.8], [4.7, 5.2],   # cluster 1
    [9.0, 1.0], [9.2, 0.8], [8.8, 1.3],   # cluster 2
])
labels = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])

scores = silhouette_score(X, labels)
print("Point | Cluster | Silhouette")
print("-" * 35)
for i, (sc, lbl) in enumerate(zip(scores, labels)):
    bar = "#" * int((sc + 1) * 10)
    print("  p" + str(i) + "  |    " + str(lbl) + "    | " +
          str(round(sc, 3)) + "  " + bar)

print()
mean_sc = np.mean(scores)
print("Mean silhouette score: " + str(round(mean_sc, 4)))
quality = "Excellent" if mean_sc > 0.7 else ("Good" if mean_sc > 0.5 else "Weak")
print("Cluster quality:       " + quality)

Cluster Validation: Internal, External, and Relative Criteria

1Internal Validation: No Labels Required

2External Validation: Comparing to Ground Truth

3Relative Validation: Comparing Different Runs

Recall check