Numerical Differentiation and Integration
CSIR NET numerical analysis: difference formulas for derivatives, Richardson extrapolation, trapezoidal, Simpson, Newton-Cotes, Gauss quadrature and errors.
- 19 min
On this page · 12 sections
Prerequisites
- Finite Differences and Interpolation: every formula on this page is “differentiate or integrate the interpolating polynomial”, and the error terms come from the interpolation error formula.
- Riemann, Improper and Lebesgue Integration: composite rules are Riemann-type sums, and the mean value theorem for integrals turns sums of error terms into one \( f^{(k)}(\xi) \).
- Continuity, Differentiability and Bounded Variation: Taylor’s theorem with remainder.
The Hook: Smaller Step, Worse Answer
Numerical Differentiation
Difference quotients from Taylor’s theorem
With \( f \) smooth enough and \( \xi \) a point near \( x \):
| Formula | Approximates | Exact relation | Order |
|---|---|---|---|
| Forward \( \frac{f(x + h) - f(x)}{h} \) | \( f'(x) \) | \( f'(x) = \frac{f(x + h) - f(x)}{h} - \frac{h}{2}f''(\xi) \) | \( O(h) \) |
| Backward \( \frac{f(x) - f(x - h)}{h} \) | \( f'(x) \) | \( f'(x) = \frac{f(x) - f(x - h)}{h} + \frac{h}{2}f''(\xi) \) | \( O(h) \) |
| Central \( \frac{f(x + h) - f(x - h)}{2h} \) | \( f'(x) \) | \( f'(x) = \frac{f(x + h) - f(x - h)}{2h} - \frac{h^2}{6}f'''(\xi) \) | \( O(h^2) \) |
| Second central \( \frac{f(x + h) - 2f(x) + f(x - h)}{h^2} \) | \( f''(x) \) | \( f''(x) = \frac{f(x + h) - 2f(x) + f(x - h)}{h^2} - \frac{h^2}{12}f^{(4)}(\xi) \) | \( O(h^2) \) |
| Three-point forward \( \frac{-3f(x) + 4f(x + h) - f(x + 2h)}{2h} \) | \( f'(x) \) | error \( \frac{h^2}{3}f'''(\xi) \) | \( O(h^2) \) |
Why the central difference wins. In \( f(x \pm h) = f \pm hf' + \frac{h^2}{2}f'' \pm \frac{h^3}{6}f''' + \cdots \), the \( f'' \) terms cancel on subtraction. Similarly the odd terms cancel in the second-difference formula.
For \( f = e^x \), \( x = 0 \), \( h = 0.1 \): forward gives 1.05171 (error \( \approx 0.052 \approx \frac{h}{2} \)), central gives 1.001668 (error \( \approx 0.0017 \approx \frac{h^2}{6} \)), and the second difference gives 1.000834 for \( f''(0) = 1 \) (error \( \approx \frac{h^2}{12} \)).
Derivatives from a difference table
From \( E = e^{hD} \), \( hD = \log(1 + \Delta) = -\log(1 - \nabla) \):
$$hf'(x_0) = \Delta f_0 - \frac{\Delta^2 f_0}{2} + \frac{\Delta^3 f_0}{3} - \frac{\Delta^4 f_0}{4} + \cdots, \qquad h^2 f''(x_0) = \Delta^2 f_0 - \Delta^3 f_0 + \frac{11}{12}\Delta^4 f_0 - \cdots$$$$hf'(x_n) = \nabla f_n + \frac{\nabla^2 f_n}{2} + \frac{\nabla^3 f_n}{3} + \cdots$$Use the forward series at the top of a table and the backward series at the bottom. Example: from \( f = 1, 8, 27, 64 \) at \( x = 1, 2, 3, 4 \) (\( \Delta f_0 = 7 \), \( \Delta^2 f_0 = 12 \), \( \Delta^3 f_0 = 6 \)): \( f'(1) \approx 7 - 6 + 2 = 3 \), exact for \( x^3 \).
Truncation versus rounding
If each computed value of \( f \) has an error of at most \( \varepsilon \), the forward-difference error is at most
$$E(h) = \frac{2\varepsilon}{h} + \frac{M_2 h}{2}, \qquad M_2 = \max\lvert f'' \rvert,$$minimised at \( h^* = 2\sqrt{\varepsilon / M_2} \). For the central difference, \( E(h) = \frac{\varepsilon}{h} + \frac{M_3 h^2}{6} \) is minimised at \( h^* = (3\varepsilon / M_3)^{1/3} \). So the best step for a first derivative is about \( \sqrt{\varepsilon} \) (forward) or \( \varepsilon^{1/3} \) (central), not “as small as possible”. Numerical differentiation is ill-conditioned; numerical integration is not.
Richardson extrapolation
If an approximation satisfies \( A(h) = A + ch^p + O(h^{p+1}) \) (or \( O(h^{p+2}) \) when only even powers appear), then
$$\frac{2^p A(h/2) - A(h)}{2^p - 1}$$cancels the \( h^p \) term. For the central difference (\( p = 2 \), even powers only): \( \frac{4D(h/2) - D(h)}{3} \) is \( O(h^4) \). With \( e^x \) at 0, \( h = 0.1 \): \( D(0.1) \approx 1.0016675 \), \( D(0.05) \approx 1.0004167 \), and the extrapolated value is \( 0.99999979 \), an error of \( 2 \times 10^{-7} \).
Numerical Integration: Newton-Cotes Rules
Integrate the interpolating polynomial on equally spaced nodes \( x_k = a + kh \).
The basic (simple) rules
| Rule | Nodes | Formula | Error \( \int - Q \) | Degree of precision |
|---|---|---|---|---|
| Midpoint | 1 (open) | \( 2h\,f(x_1) \) on \( [x_0, x_2] \); or \( h\,f(m) \) on width \( h \) | \( +\frac{h^3}{24}f''(\xi) \) (width \( h \)) | 1 |
| Trapezoidal | 2 | \( \frac{h}{2}(f_0 + f_1) \) | \( -\frac{h^3}{12}f''(\xi) \) | 1 |
| Simpson’s \( \frac{1}{3} \) | 3 | \( \frac{h}{3}(f_0 + 4f_1 + f_2) \) | \( -\frac{h^5}{90}f^{(4)}(\xi) \) | 3 |
| Simpson’s \( \frac{3}{8} \) | 4 | \( \frac{3h}{8}(f_0 + 3f_1 + 3f_2 + f_3) \) | \( -\frac{3h^5}{80}f^{(4)}(\xi) \) | 3 |
| Boole’s | 5 | \( \frac{2h}{45}(7f_0 + 32f_1 + 12f_2 + 32f_3 + 7f_4) \) | \( -\frac{8h^7}{945}f^{(6)}(\xi) \) | 5 |
| Weddle’s | 7 | \( \frac{3h}{10}(f_0 + 5f_1 + f_2 + 6f_3 + f_4 + 5f_5 + f_6) \) | (not a pure Newton-Cotes rule) | 5 |
Finding a rule’s weights. Demand exactness for \( 1, x, x^2, \dots \) in turn. Example: \( \int_{-1}^{1} f \approx \alpha f(-1) + \beta f(0) + \gamma f(1) \). Exact for \( 1, x, x^2 \): \( \alpha + \beta + \gamma = 2 \), \( -\alpha + \gamma = 0 \), \( \alpha + \gamma = \frac{2}{3} \). So \( \alpha = \gamma = \frac{1}{3} \), \( \beta = \frac{4}{3} \): Simpson’s rule, recovered.
Composite rules
Divide \( [a, b] \) into \( n \) subintervals of width \( h = \frac{b - a}{n} \).
$$\textbf{Composite trapezoidal:}\quad T_n = \frac{h}{2}\Big[f_0 + 2(f_1 + f_2 + \dots + f_{n-1}) + f_n\Big], \qquad \int_a^b f - T_n = -\frac{(b - a)h^2}{12}f''(\xi).$$$$\textbf{Composite Simpson's 1/3 } (n \text{ even}):\quad S_n = \frac{h}{3}\Big[f_0 + 4(f_1 + f_3 + \dots + f_{n-1}) + 2(f_2 + f_4 + \dots + f_{n-2}) + f_n\Big],$$$$\int_a^b f - S_n = -\frac{(b - a)h^4}{180}f^{(4)}(\xi).$$Composite Simpson’s \( \frac{3}{8} \) needs \( n \) a multiple of 3; error \( -\frac{(b - a)h^4}{80}f^{(4)}(\xi) \).
| Composite rule | Global error | Halving \( h \) divides the error by about |
|---|---|---|
| Trapezoidal | \( O(h^2) \) | 4 |
| Simpson’s \( \frac{1}{3} \) or \( \frac{3}{8} \) | \( O(h^4) \) | 16 |
| Boole’s | \( O(h^6) \) | 64 |
Signs and convexity
Since \( \int - T = -\frac{h^3}{12}f''(\xi) \) and \( \int - M = +\frac{h^3}{24}f''(\xi) \): if \( f'' > 0 \) (convex) on \( [a, b] \), the trapezoidal rule overestimates and the midpoint rule underestimates, and the midpoint error is about half the trapezoidal error with the opposite sign. Combining them to cancel the leading error,
$$\frac{2M + T}{3} = \text{Simpson's rule on } [a, b].$$If \( f'' \) changes sign on \( [a, b] \), no such conclusion holds. (For \( \frac{1}{1 + x^2} \) on \( [0, 1] \), \( f'' \) changes sign at \( \frac{1}{\sqrt 3} \), and the trapezoidal value above happens to underestimate.)
Romberg integration
The composite trapezoidal error has an expansion in even powers of \( h \) (Euler-Maclaurin): \( T(h) = I + c_1h^2 + c_2h^4 + \cdots \) for smooth \( f \). Richardson extrapolation applies repeatedly:
$$R_{k,1} = \frac{4T(h/2) - T(h)}{3} \ (\text{this is Simpson's rule}), \qquad R_{k,j} = \frac{4^j R_{k,j-1} - R_{k-1,j-1}}{4^j - 1}.$$A side effect of the same expansion: for a smooth periodic function integrated over a full period, all the \( c_j \) vanish and the trapezoidal rule converges extremely fast.
Gaussian Quadrature
Newton-Cotes fixes the nodes and chooses only the weights. Choosing both nodes and weights, \( n \) points give \( 2n \) free parameters, enough to integrate polynomials of degree \( 2n - 1 \).
| \( n \) | Nodes \( t_i \) | Weights \( w_i \) | Exact up to degree |
|---|---|---|---|
| 1 | 0 | 2 | 1 (midpoint rule) |
| 2 | \( \pm\frac{1}{\sqrt 3} \) | 1, 1 | 3 |
| 3 | \( 0, \pm\sqrt{\frac{3}{5}} \) | \( \frac{8}{9}, \frac{5}{9}, \frac{5}{9} \) | 5 |
Change of interval. For \( \int_a^b f(x)\,dx \), substitute \( x = \frac{b - a}{2}t + \frac{a + b}{2} \):
$$\int_a^b f(x)\,dx = \frac{b - a}{2}\int_{-1}^{1} f\Big(\frac{b - a}{2}t + \frac{a + b}{2}\Big)dt.$$Example: two-point Gauss for \( \int_0^1 e^x\,dx \) gives about 1.717896 against the exact \( e - 1 \approx 1.718282 \), using only two function values. The two-point rule fails first on \( x^4 \): it gives \( 2 \cdot \frac{1}{9} = \frac{2}{9} \) for \( \int_{-1}^1 t^4\,dt = \frac{2}{5} \).
Memory Tricks
Don’t Confuse
| This | vs | That |
|---|---|---|
| Local (single-panel) error: Simpson \( O(h^5) \) | global (composite) error: Simpson \( O(h^4) \) | |
| Simpson’s \( \frac{1}{3} \): \( n \) even | Simpson’s \( \frac{3}{8} \): \( n \) a multiple of 3 | |
| Both Simpson rules: degree of precision 3 | 3/8 rule has the larger error constant (\( \frac{3}{80} > \frac{1}{90} \) per panel) | |
| Newton-Cotes: fixed equally spaced nodes, \( n \) points give degree \( n - 1 \) or \( n \) | Gauss: optimal nodes, \( n \) points give degree \( 2n - 1 \) | |
| Differentiation: rounding error grows as \( h \to 0 \) | integration: rounding stays bounded as \( h \to 0 \) | |
| Trapezoid on convex \( f \): overestimate | midpoint on convex \( f \): underestimate |
Common Mistakes
Practice
All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.
Level 1: Foundation
- State the composite trapezoidal rule and its error term.
- What is the degree of precision of Simpson’s \( \frac{3}{8} \) rule?
- Write the two-point Gauss-Legendre rule on \( [-1, 1] \).
Answers
- \( T_n = \frac{h}{2}[f_0 + 2(f_1 + \dots + f_{n-1}) + f_n] \); \( \int_a^b f - T_n = -\frac{(b - a)h^2}{12}f''(\xi) \).
- 3 (the same as Simpson’s \( \frac{1}{3} \)).
- \( \int_{-1}^{1} f \approx f\left(-\frac{1}{\sqrt 3}\right) + f\left(\frac{1}{\sqrt 3}\right) \), exact for cubics.
Level 2: Part B Style
Practice Q1. Simpson’s \( \frac{1}{3} \) rule with \( h = 1 \) applied to \( \int_0^2 x^4\,dx \) gives
(a) 6.4 (b) \( \frac{20}{3} \) (c) 8 (d) \( \frac{19}{3} \)
Answer: (b)
\( \frac{1}{3}(0 + 4 \cdot 1 + 16) = \frac{20}{3} \approx 6.667 \). The exact value is \( \frac{32}{5} = 6.4 \), and the error \( 6.4 - \frac{20}{3} = -\frac{4}{15} \) agrees with \( -\frac{h^5}{90}f^{(4)} = -\frac{24}{90} = -\frac{4}{15} \) (here \( f^{(4)} = 24 \) is constant).
Practice Q2. The smallest number of subintervals for which the composite trapezoidal error bound for \( \int_0^1 e^{-x^2}\,dx \) is at most \( 10^{-4} \) is
(a) 21 (b) 29 (c) 41 (d) 58
Answer: (c)
\( \max_{[0,1]}\lvert f'' \rvert = 2 \), so the bound is \( \frac{h^2}{6} \le 10^{-4} \), \( h \le 0.0245 \), \( n \ge 40.8 \): \( n = 41 \).
Practice Q3. The rule \( \int_0^1 f(x)\,dx \approx a f(0) + b f(c) \) is to have the highest possible degree of precision. Then
(a) \( a = \frac{1}{4}, b = \frac{3}{4}, c = \frac{2}{3} \) (b) \( a = \frac{1}{2}, b = \frac{1}{2}, c = 1 \) (c) \( a = \frac{1}{3}, b = \frac{2}{3}, c = \frac{3}{4} \) (d) \( a = 0, b = 1, c = \frac{1}{2} \)
Answer: (a)
Exactness for \( 1, x, x^2 \): \( a + b = 1 \), \( bc = \frac{1}{2} \), \( bc^2 = \frac{1}{3} \). Dividing, \( c = \frac{2}{3} \), so \( b = \frac{3}{4} \), \( a = \frac{1}{4} \). For \( x^3 \): \( bc^3 = \frac{3}{4}\cdot\frac{8}{27} = \frac{2}{9} \ne \frac{1}{4} \), so the degree of precision is 2. Options (b) (trapezoid) and (d) (midpoint) have degree 1.
Practice Q4. The central difference approximation \( D(h) = \frac{f(x + h) - f(x - h)}{2h} \) has error \( c_1h^2 + c_2h^4 + \cdots \). The combination that has error \( O(h^4) \) is
(a) \( 2D(h/2) - D(h) \) (b) \( \frac{4D(h/2) - D(h)}{3} \) (c) \( \frac{D(h) + D(h/2)}{2} \) (d) \( \frac{16D(h/2) - D(h)}{15} \)
Answer: (b)
\( D(h/2) = f' + \frac{c_1h^2}{4} + \cdots \). Then \( 4D(h/2) - D(h) = 3f' + O(h^4) \). Option (a) is the extrapolation for an \( O(h) \) method; option (d) is the next Romberg step, which would cancel \( h^4 \) after \( h^2 \) is gone.
Level 3: Part C Style
Practice Q5. For \( I = \int_0^1 \frac{dx}{1 + x^2} \) with \( h = 0.25 \), the composite trapezoidal value is \( T \approx 0.782794 \) and the composite Simpson value is \( S \approx 0.785392 \). Which statements are true?
(a) \( 4S \) agrees with \( \pi \) to four decimal places (after rounding). (b) Since \( f(x) = \frac{1}{1 + x^2} \) is convex on \( [0, 1] \), \( T \) must overestimate \( I \). (c) \( S = \frac{4T_4 - T_2}{3} \), where \( T_2 \), \( T_4 \) are the trapezoidal values with 2 and 4 subintervals. (d) The composite Simpson error bound \( \frac{(b - a)h^4}{180}\max\lvert f^{(4)} \rvert \) shrinks by a factor of 16 if \( h \) is halved.
Answer: (a), (c) and (d)
- (a) True. \( 4S \approx 3.141569 \), which rounds to 3.1416, as does \( \pi \approx 3.14159 \).
- (b) False. \( f''(x) = \frac{6x^2 - 2}{(1 + x^2)^3} \) is negative on \( [0, \frac{1}{\sqrt 3}) \) and positive after, so \( f \) is not convex on \( [0, 1] \). In fact \( T < I \approx 0.785398 \).
- (c) True. This is the first Romberg step: Richardson extrapolation of the trapezoidal rule gives Simpson’s rule on the finer grid. (Check: \( T_2 = \frac{0.5}{2}(1 + 1.6 + 0.5) = 0.775 \), and \( \frac{4(0.782794) - 0.775}{3} \approx 0.785392 \).)
- (d) True. The bound is proportional to \( h^4 \).
Practice Q6. Which statements about quadrature rules are true?
(a) Simpson’s \( \frac{1}{3} \) rule and Simpson’s \( \frac{3}{8} \) rule have the same degree of precision. (b) The \( n \)-point Gauss-Legendre rule is exact for all polynomials of degree \( 2n - 1 \) and for no polynomial of degree \( 2n \). (c) The two-point Gauss-Legendre rule integrates \( t^4 \) exactly on \( [-1, 1] \). (d) Boole’s rule has degree of precision 5.
Answer: (a), (b) and (d)
- (a) True. Both have degree 3.
- (b) True. Exactness up to degree \( 2n - 1 \) is the defining property. For degree \( 2n \), let \( E(p) = \int_{-1}^1 p - \sum w_i p(t_i) \). If \( p = a t^{2n} + (\text{lower terms}) \) with \( a \ne 0 \), then \( E(p) = a\,E(t^{2n}) \), because the lower terms are integrated exactly. And \( E(t^{2n}) = E(\omega^2) \), where \( \omega(t) = \prod_i (t - t_i) \) is monic of degree \( n \): \( E(\omega^2) = \int_{-1}^1 \omega^2 - 0 > 0 \). So \( E(p) \ne 0 \) for every polynomial of exact degree \( 2n \). (For \( n = 1 \): the midpoint rule gives \( 2c \) for \( \int_{-1}^1 (at^2 + bt + c)\,dt = \frac{2a}{3} + 2c \).)
- (c) False. It gives \( 2 \cdot \frac{1}{9} = \frac{2}{9} \), not \( \frac{2}{5} \).
- (d) True. It is the 5-point closed Newton-Cotes rule (\( n = 4 \), even), so precision \( n + 1 = 5 \).
Practice Q7. For \( f(x) = e^x \) at \( x = 0 \) with \( h = 0.1 \), a student obtains the forward difference \( 1.05171 \) and the central difference \( 1.00167 \). Which statements are true?
(a) The forward-difference error is close to \( \frac{h}{2}f''(0) = 0.05 \). (b) The central-difference error is close to \( \frac{h^2}{6}f'''(0) \approx 0.00167 \). (c) Halving \( h \) would reduce the central-difference error by a factor of about 2. (d) If each function value carries rounding error up to \( \varepsilon \), the total central-difference error is minimised for \( h \) of order \( \varepsilon^{1/3} \), not as \( h \to 0 \).
Answer: (a), (b) and (d)
- (a) True. \( 1.05171 - 1 = 0.0517 \approx 0.05 \) (the next Taylor term \( \frac{h^2}{6} \approx 0.0017 \) accounts for the rest).
- (b) True. \( 1.00167 - 1 = 0.00167 \approx \frac{0.01}{6} \).
- (c) False. The error is \( O(h^2) \): halving \( h \) divides it by about 4 (\( D(0.05) - 1 \approx 0.000417 \)).
- (d) True. Total error \( \approx \frac{\varepsilon}{h} + \frac{M_3h^2}{6} \), minimised at \( h = (3\varepsilon/M_3)^{1/3} \).
Quick Revision Table
| Formula | Error |
|---|---|
| Forward \( \frac{f(x+h) - f(x)}{h} \) | \( O(h) \): \( -\frac{h}{2}f'' \) |
| Central \( \frac{f(x+h) - f(x-h)}{2h} \) | \( O(h^2) \): \( -\frac{h^2}{6}f''' \) |
| \( \frac{f(x+h) - 2f(x) + f(x-h)}{h^2} \) | \( O(h^2) \): \( -\frac{h^2}{12}f^{(4)} \) |
| \( hD = \log(1 + \Delta) \) | \( hf'_0 = \Delta - \frac{\Delta^2}{2} + \frac{\Delta^3}{3} - \cdots \) |
| Trapezoid, one panel / composite | \( -\frac{h^3}{12}f'' \) / \( -\frac{(b-a)h^2}{12}f'' \) |
| Simpson 1/3, one panel / composite | \( -\frac{h^5}{90}f^{(4)} \) / \( -\frac{(b-a)h^4}{180}f^{(4)} \) |
| Simpson 3/8, one panel | \( -\frac{3h^5}{80}f^{(4)} \) |
| Midpoint, width \( h \) | \( +\frac{h^3}{24}f'' \) |
| Degrees of precision | midpoint 1, trapezoid 1, Simpson 3, 3/8 3, Boole 5, Weddle 5, Gauss \( n \)-pt \( 2n - 1 \) |
| Gauss 2-pt / 3-pt | \( \pm\frac{1}{\sqrt 3} \), weights 1 / \( 0, \pm\sqrt{3/5} \), weights \( \frac{8}{9}, \frac{5}{9} \) |
| Richardson (even powers) | \( \frac{4A(h/2) - A(h)}{3} \); Romberg step 1 = Simpson |
Key Takeaways
What’s Next
- Picard, Euler, Modified Euler and Runge-Kutta: solving \( y' = f(x, y) \) is integrating along the solution; Euler is the rectangle rule, Heun is the trapezoidal rule, and RK4 reduces to Simpson’s rule when \( f \) depends on \( x \) alone.
- Finite Differences and Interpolation: the source of every formula here.
- Numerical Analysis: Quick Revision.