Numerical Analysis

A computer calculation usually involves rounded numbers and approximations. Numerical analysis asks how much these choices change the answer and how to keep the error within a useful tolerance. The main topics are:

1. floating-point representation and rounding error;
2. conditioning of problems and stability of algorithms;
3. root finding and optimization;
4. interpolation and approximation;
5. numerical differentiation and integration;
6. linear systems and numerical ODE solutions.

For a course with worked methods and problem sets, use MIT OpenCourseWare 18.330.

A minimal error vocabulary

Start by distinguishing the size of an error from what caused it. For an exact value $y$ and approximation $\hat y$, absolute error is $|\hat y-y|$; relative error is $|\hat y-y|/|y|$ when $y\ne0$. Near zero, an absolute tolerance is essential.

Conditioning describes sensitivity of the mathematical problem to input perturbations. For $y=a-b$, if each input has relative uncertainty at most $\varepsilon$, then the output’s absolute uncertainty is at most $\varepsilon(|a|+|b|)$. Its relative bound is that quantity divided by $|a-b|$, which becomes large when the inputs nearly agree. A better algorithm cannot recover information absent from the inputs.

Eigenvalues and SVD develops the corresponding matrix example: small singular values amplify perturbations, and normal equations square the spectral condition number.

Stability describes how an algorithm propagates rounding and other computational errors. A backward-stable algorithm returns the exact answer to a nearby input problem; forward accuracy still depends on conditioning. Finite-precision arithmetic also makes algebraic rearrangement consequential: with binary64 arithmetic, (1e16 + 1) - 1e16 evaluates to 0.0, not the real-arithmetic value 1.

Discretization error comes from replacing a continuous problem with finitely many samples. Refining the grid can reduce it without reducing data uncertainty or roundoff. The numerical integration note gives a complete first example: formulas, executable code, smoothness assumptions, error bounds, and a separate bound for truncating an infinite tail.

For a calculation, specify the desired absolute/relative tolerance, check domain and conditioning, then compare refinements with an analytic bound when available. A small change between successive approximations is evidence, not proof of accuracy.

Run the short examples in Python’s floating-point tutorial to compare stored values, displayed decimals, and approximate equality. They connect the rounding discussion here to practical checks with math.isclose and accurate summation.

My reading · Opened notes are remembered in this browser.