What the scalers do
Switch to StandardScaler. Each column has its mean subtracted and is divided by its standard deviation, so both end up centred on 0 with a spread of about 1. The scatter becomes roughly circular, both columns contribute comparably, and the accuracy in the readout moves.
z = (x - mean) / std
Switch to MinMaxScaler. Each column is mapped onto [0, 1] using its minimum and maximum:
x' = (x - min) / (max - min)
Both fix the units problem. They differ in how they handle extremes, and that is the basis for choosing.
| StandardScaler | MinMaxScaler |
|---|
| Output range | unbounded, roughly −3 to 3 | exactly [0, 1] |
| Uses | mean, standard deviation | minimum, maximum |
| One extreme value | shifts the mean a little | compresses everything else |
| Good for | most things, especially with outliers | bounded inputs, image pixels, neural nets |
MinMax's weakness is worth stating plainly. Both statistics it uses are the most outlier-sensitive numbers available. One salary of ten million maps to 1.0 and crushes every real salary into the bottom half of a per cent of the range.
Which models care
Sensitive, because they measure distance or magnitude:
- k-NN and k-means — distance is the entire algorithm
- SVM with an RBF or polynomial kernel — the kernel is a distance
- PCA — it finds directions of maximum variance, and unscaled variance is dominated by whichever column has the biggest units
- Neural networks — not for correctness, but gradients on wildly different scales make optimisation slow and unstable
- Ridge and Lasso — the penalty is on coefficient size, and coefficient size depends on the units of its column
Not sensitive:
- Decision trees, random forests, gradient boosting
That last group surprises people, and the reason is worth having. A tree split asks is income > 45000. Rescale the column and the question becomes is income' > 0.53, which partitions the rows into exactly the same two groups. Any monotonic transformation leaves every possible split unchanged, so the tree is identical. Scaling before a random forest is not wrong; it is simply a no-op.
The rule that matters more than the choice
Fit the scaler on the training data only, then apply it to the test data without refitting.
The mean and standard deviation are learned parameters. Computing them from the whole dataset lets information about the test set reach the model before it is evaluated, and the reported score stops being an estimate of anything. That is [data leakage](data_leakage.html), it is the most common serious mistake in applied machine learning, and it is why scalers belong inside a [pipeline](ml_pipelines.html) rather than being applied by hand.
Standardising, and why trees do not care
The fix is one formula applied per column, z = (x - mean) / std, and working it shows why some models transform completely and others not at all. Age spanning 25-55 and income spanning 20,000-70,000, standardised:
| raw | z-score |
|---|
| age 30 (mean 40, sd 10) | −1.0 |
| age 55 | +1.5 |
| income 25,000 (mean 45k, sd 15k) | −1.3 |
| income 70,000 | +1.7 |
Both columns now live on the same −3-to-3 scale, so a year of age and a standard-deviation of income count comparably in a Euclidean distance — where before, the income term's raw spread of 50,000 against age's 30 made age numerically invisible to k-NN, k-means, SVM-RBF and PCA. A decision tree, by contrast, is completely unmoved: its split asks is income > 45000, which after scaling becomes is income' > 0.53, and that partitions the rows into exactly the same two groups. Any monotonic rescaling leaves every possible split unchanged, so scaling before a random forest is a genuine no-op — not wrong, just wasted effort — while for the distance-based models it is the difference between a usable model and a one-dimensional one.
The choice between the two common scalers comes down to outliers. StandardScaler uses the mean and standard deviation, so one extreme value shifts the centre only a little; MinMaxScaler uses the minimum and maximum, the two most outlier-sensitive numbers there are, so a single salary of ten million maps to 1.0 and crushes every real salary into the bottom fraction of a per cent of the range. StandardScaler is the safer default for most data, MinMax suits genuinely bounded inputs like image pixels. One thing neither should touch is a one-hot column: it is already 0 and 1, and standardising it turns two meaningful values into two arbitrary ones for no gain.
And if you scale the target as well as the features, remember to invert the transform on the predictions, or the error metric comes back in scaled units and means nothing — a small bookkeeping step that is easy to forget and quietly corrupts every reported number.
Where it goes wrong
Fitting the scaler on everything. The number you report goes up and the model does not get better.
Scaling the target variable and forgetting to invert it. Predictions come back in scaled units and the error metric is meaningless.
MinMax on data with outliers. The real data ends up in a sliver of the range.
Scaling one-hot columns. They are already 0 and 1. StandardScaler turns them into two arbitrary values and makes the model harder to read for no gain.
Which models care, and which do not
One dataset, one column rescaled by 10,000, five models. Two collapse, two do not move at all, and the fifth is the interesting case.