Harness the tangent line operator to double precision digits on every iteration and reach one thousand digits in eleven steps.
- Step the tangent projection to watch residual errors square
- Compare the quadratic doubling rate against linear bisection
- Drive the iterate near inflection points to witness flat-slope divergence
After thisYou can solve non-linear roots and compute fast reciprocal square roots with quadratic convergence.
Short on timeThe IterationLab interactive panel in §1
The tangent line that doubles your digits
3 minthe hook
Eleven Newton steps reach 1,000 correct digits. Bisection needs 3,322 steps. A contraction with needs 23,025 steps per digit, so the derivative is what buys the doubling.
In the plague year of 1669, Isaac Newton calculated roots at Woolsthorpe Manor by replacing each curve with its local linear shadow and following that line to the axis.
The picture is the tangent line: draw it at the current guess and take its axis crossing as the next guess. The panel opens on at .
Before you touch the transport, commit to a prediction below.
f(x) = x² − 2 · f′(x) = 2x · g(x) = (x + 2)/(x + 1) · true root √2 = 1.414213562373
| n | x_n | f(x_n) | e_n | e_{n+1} / e_n² |
|---|---|---|---|---|
| 0 | 1.2 | -0.56 | 0.2142136 | — |
Order of convergence p = 2. The ratio column settles on a constant exactly when p is the right order; here that constant is 0.35355. A ratio that keeps drifting means the step is not yet in the asymptotic regime.
Two roots, ±√2. g is contracting at +√2 (|g′| = 0.1716) and repelling at −√2 (|g′| = 5.83), so the same map can converge or run away depending on where you start.
Curve, current guess, tangent, axis. Step it and watch the tangent's crossing become the next guess.
At step n, your estimate of √2 has error e_n = 0.001 (accurate to 3 decimal places). How accurate will the next single step be?
Press step once and watch the columns update. The e_n column tracks distance to the true root ; the ratio column divides the new error by the square of the previous error. Watch that ratio: it should settle near 0.3536 for , while a linear method’s ratio would settle near a constant.
NextWhy does bisection require thousands of steps where the tangent line takes eleven?
The algebraic recipe versus the geometric truth
2 minthe trap
Textbooks often introduce root-finding as an imperative loop: compute until .
That algebraic formula tells you what to calculate, but obscures what the step actually is. Treating the update as a formula hides both why it converges so furiously and why it can violently diverge.
NextHow does the derivative construct a local linear model of the target function?
The Newton map and its superattracting fixed points
2 minthe invariant
Construct the tangent line to at the current point :
The root of this tangent line defines the next guess. Setting yields , recovering the algebraic rule from basic geometry.
This defines the Newton iteration map . Fixed points where correspond precisely to roots where .
Differentiating the iteration map clarifies its dynamic behavior:
At any simple root with , the derivative vanishes: . In dynamical systems, a fixed point with zero derivative is superattracting: iterates do not merely approach the root, they are sucked in with quadratic velocity (Fatou 1918, Julia 1918).
NextWhat superattracting fixed point causes digits of precision to double every step?
Quadratic convergence: why digits double at every step
2 minthe algebra
Let be a simple root and define error . Expanding and in Taylor series around :
Substitute both expansions into the Newton update rule. The first-order linear terms cancel out completely:
The subsequent error is proportional to the square of the previous error. For at , the asymptotic error ratio settles on .
This squaring means precision compounds exponentially: 2 correct digits become 4, then 8, 16, 32, and 64.
1 minboss round
Wallis's polynomial f(x) = x³ - 2x - 5 has its real root near x ≈ 2.0946. If you start Newton's method at x₀ = 0.82 (where f'(0.82) = 0.0172), what happens on step 1?
NextWhat happens when the iterate wanders near a local extremum with zero derivative?
Driving the lab: tangents, secants, and the abyss of flat slopes
2 minthe simulation
Switch the interactive lab to Bisection. The geometry shifts from dynamic tangent projections to shrinking brackets, with the error ratio locked at . Bisection provides order : safe and guaranteed, but linearly plodding.
Now switch to the Secant method. Instead of computing exact derivatives, it approximates tangents using chords through the last two guesses, delivering golden-ratio order .
Switch to and drag the starting point near , where . The tangent line flattens out, casting the iterate into the mathematical wilderness.
| n | log n | note |
|---|---|---|
| 2⁸ | 8 | 2 steps |
| 2¹⁶ | 16 | 3 steps |
| 2³² | 32 | 4 steps |
| 2⁶⁴ | 64 | 5 steps |
| 2¹²⁸ | 128 | 6 steps |
| 2²⁵⁶ | 256 | 7 steps |
Precision is n = 2^b bits. Reaching 256 bits of precision requires only seven steps from x₀ = 1.2.
Emphasised column: log n · highlighted row: n = 2⁶⁴
Advancing from 8-bit precision to 256-bit precision scales accuracy by , yet demands only five additional iterations. The computational cost is logarithmic in the number of accurate bits.
NextHow does Newton's iteration accelerate square roots in 3D graphics hardware?
The engine of numerical computing: from fast inverse square roots to ML
2 minsystems
Newton’s method forms the bedrock of modern numerical algorithms:
- Fast reciprocal square root. Computing relies on a single Newton step applied to , operating entirely with multiplications and subtractions without division.
- Multivariate optimization. In machine learning and scientific computing, finding extrema of loss functions solves using Newton-Raphson, stepping along with the Hessian matrix.
- Hybrid root solvers. Production libraries run Brent’s method (Brent 1973), which combines the guaranteed convergence of bisection with the superlinear speed of secant and inverse quadratic interpolation.
The fractal frontier: Cayley’s problem and Julia sets
In 1879, Arthur Cayley investigated Newton’s method on the complex cubic . He attempted to map which initial starting values converge to which of the three cube roots of unity.
The answer revealed one of the foundational objects of chaos theory. The boundaries separating the three basins of attraction are nowhere smooth: they form an infinitely intricate fractal known as the Julia set.
On the boundaries between basins, Newton iterations bounce chaotically forever. In production numerical software, trust-region protections and line-search backtracking ensure iterations stay anchored within legitimate basins of attraction.
Historical provenance and attributions
- Newton 1669 — De analysi per aequationes numero terminorum infinitas: introduced the geometric tangent root-finding construction on polynomials.
- Raphson 1690 — Analysis aequationum universalis: simplified Newton’s approach into the modern algebraic recurrence.
- Simpson 1740 — Generalized the method to general nonlinear and transcendental equations using fluxions and derivatives.
- Taylor 1715 — Methodus Incrementorum Directa et Inversa: provided the polynomial expansion proving local quadratic convergence .
- Fatou 1918 and Julia 1918 — Developed the theory of rational iteration in the complex plane, characterizing superattracting fixed points and fractal basin boundaries.
- Brent 1973 — Algorithms for Minimization without Derivatives: constructed the reliable hybrid root-finding method combining bisection and secant steps.
- Smale 1986 — Newton’s Method Estimates from Data at One Point: established point-estimate -theory certifying convergence from single initial values.