Taylor Series
Approximate any smooth curve by a polynomial that agrees with it at one point. Add terms and watch the agreement spread.
Overview
The construction
Pick a point. Build a polynomial that agrees with the function there, then also agrees with its slope, then its curvature, then the next derivative, and so on.
f(x) ~ f(a) + f'(a)(x-a) + f''(a)(x-a)^2/2! + f'''(a)(x-a)^3/3! + ...Each term pins down one more derivative at a. The factorials are there because differentiating (x-a)^k repeatedly produces k!, and dividing by it cancels exactly.
Set the terms control to 0 and step it upward. At 0 you have a horizontal line at the right height. At 1, the tangent. At 2, a parabola that also has the right bend. Each step hugs the curve a little further out.
Taylor Series
Worth knowing
Taylor Series
Replacing a hard function with a polynomial, and being precise about where that is allowed.
Local, and only local
The readout gives how far the approximation stays within 0.1 of the truth, and this is the number worth watching.
More terms extend the range. They do not make it infinite, and the extension gets slower. The polynomial agrees perfectly at the expansion point and gets worse with distance, always — the whole construction is built from information at a single point, so it cannot know anything about elsewhere.
Move the expansion point and the region of agreement moves with it. That is the right mental model: a Taylor series is a statement about a neighbourhood.
Where it earns its keep
Second-order optimisation. Newton's method is exactly the two-term expansion of the loss: approximate it as a quadratic near the current point, jump to that quadratic's minimum, repeat. Everything in [the Hessian module](jacobian_and_hessian.html) is this idea.
Small-angle approximations. sin(x) ~ x for small x is the one-term expansion, and it is why pendulum equations are solvable at all.
Backpropagation's justification. The chain rule is exact, but the argument that a small weight change produces a proportional loss change is a first-order expansion.
Numerical methods. Finite differences, Runge-Kutta integrators and most error bounds are derived by expanding and discarding.
exp, log, sin in a standard library. Not looked up in a table; evaluated from a truncated series with a range reduction in front of it.
Where it fails
Switch the function control to 1 / (1 + x squared).
This function is smooth everywhere on the real line — no kinks, no asymptotes, differentiable as many times as you like. Yet expand it about 0 and add terms, and beyond about x = 1 the approximation does not merely fail to improve; it gets worse with every term added.
The reason is invisible from the real line. Viewed over the complex numbers the function has poles at *i* and *−i*, distance 1 from the origin, and the radius of convergence is the distance to the nearest singularity wherever it sits. That radius is 1, and no number of real terms escapes it.
This is the Runge phenomenon, and its lesson is practical: smooth on the reals does not mean a Taylor series converges everywhere. Adding terms is not always progress.
Where it goes wrong
Extrapolating far from the expansion point. The error grows with distance, and past the radius of convergence it grows without bound.
Assuming more terms is always better. Not past the radius, and not numerically — high-order terms involve large factorials and cancelling quantities, which loses precision in floating point.
Expanding about the wrong point. Expand about where you will evaluate. A series about 0 is the wrong tool for estimating at x = 5.
Forgetting smoothness is required. The function needs derivatives of every order at the expansion point. |x| has none at 0, so there is no series there.
e to the x, worked by hand
Expand e^x about a = 0. Every derivative of e^x is e^x, and e^0 = 1, so every coefficient is 1/k! and the series is as clean as they come:
e^x ~ 1 + x + x^2/2 + x^3/6 + x^4/24 + ...Evaluate it at x = 1, where the true value is 2.718282, adding one term at a time:
| terms | value | error |
|---|---|---|
| 1 | 1.0000 | 1.7183 |
| 2 | 2.0000 | 0.7183 |
| 3 | 2.5000 | 0.2183 |
| 4 | 2.6667 | 0.0516 |
| 5 | 2.7083 | 0.0099 |
| 6 | 2.7167 | 0.0016 |
Six terms and the error is under a tenth of a percent. Now the *same six terms* at x = 3, where the truth is 20.0855:
| terms | value | error |
|---|---|---|
| 1 | 1.000 | 19.09 |
| 2 | 4.000 | 16.09 |
| 3 | 8.500 | 11.59 |
| 4 | 13.000 | 7.09 |
| 5 | 16.375 | 3.71 |
| 6 | 18.400 | 1.69 |
Still 8% out. The reason is visible in the term itself: the k-th term is x^k / k!, so the numerator grows like 3^k while the factorial does not overtake it until k passes 3. Near the expansion point the terms shrink from the start; far from it they grow first and shrink later, and you have to push through that bulge before the series is any use. Matching at x = 3 the accuracy six terms gave at x = 1 takes eleven.
That is the whole "local, and only local" claim, in numbers: the same polynomial, unchanged, is excellent at one distance and unusable at three times it. It is also why real implementations do range reduction first — e^3 is computed as e^0.25 raised to a power, not by expanding at 3.
Questions people ask
Taylor or Maclaurin? A Maclaurin series is a Taylor series with the expansion point at zero. That is the entire difference, and it is why the two words are used almost interchangeably.
How many terms do I need? Whatever makes the first term you dropped small enough. For a well-behaved alternating series the remainder is bounded by the size of that next term, which makes it a genuinely practical stopping rule. Otherwise use the Lagrange remainder, which bounds the error by the largest (n+1)-th derivative anywhere between the expansion point and where you are evaluating — and that bound is why the error grows with distance.
Why does 1/(1 + x²) break at x = 1 when it is smooth everywhere? Because the real line is the wrong place to look. Allow complex numbers and the function blows up at x = i and x = -i, both at distance 1 from the origin. A power series converges inside a *disc* reaching the nearest singularity in the complex plane, so the radius here is 1 regardless of how well behaved the function looks on the reals. The failure is real; the cause is elsewhere.
Is the Taylor polynomial the best polynomial approximation? No, and this surprises people. It is the best one *at the expansion point*, where it matches as many derivatives as it has terms. Across an interval it is usually beaten comfortably by a minimax or Chebyshev fit, which spreads the error evenly instead of concentrating accuracy at one spot. Numerical libraries use those, not Taylor, for precisely this reason.
Does a Taylor series always converge to the function? Not necessarily. The standard counterexample is e^(-1/x^2) extended by zero at the origin: every derivative there is 0, so its series is identically 0, which agrees with the function at exactly one point. Smooth is not enough; the property that makes the series work is called being *analytic*.
Rebuild a function from its derivatives
Terms added one at a time, with the error after each. The approximation is excellent near the expansion point and hopeless far from it.
Check yourself
0 of …Answer without scrolling back up.
What does the second term of a Taylor series give you?
Why does the series for 1/(1+x^2) stop improving beyond x = 1?
A Taylor series is a statement about:
Taylor Series
Pick a point. Build a polynomial that agrees with the function there, then also agrees with its slope, then its curvature, then the next derivative, and so on.