Skip to content

Numerical Differentiation and Integration

CSIR NET numerical analysis: difference formulas for derivatives, Richardson extrapolation, trapezoidal, Simpson, Newton-Cotes, Gauss quadrature and errors.

  • 19 min read
  • Updated
On this page · 12 sections
  1. Prerequisites
  2. The Hook: Smaller Step, Worse Answer
  3. Numerical Differentiation
  4. Numerical Integration: Newton-Cotes Rules
  5. Gaussian Quadrature
  6. Memory Tricks
  7. Don’t Confuse
  8. Common Mistakes
  9. Practice
  10. Quick Revision Table
  11. Key Takeaways
  12. What’s Next

Prerequisites

The Hook: Smaller Step, Worse Answer

Numerical Differentiation

Difference quotients from Taylor’s theorem

With \( f \) smooth enough and \( \xi \) a point near \( x \):

FormulaApproximatesExact relationOrder
Forward \( \frac{f(x + h) - f(x)}{h} \)\( f'(x) \)\( f'(x) = \frac{f(x + h) - f(x)}{h} - \frac{h}{2}f''(\xi) \)\( O(h) \)
Backward \( \frac{f(x) - f(x - h)}{h} \)\( f'(x) \)\( f'(x) = \frac{f(x) - f(x - h)}{h} + \frac{h}{2}f''(\xi) \)\( O(h) \)
Central \( \frac{f(x + h) - f(x - h)}{2h} \)\( f'(x) \)\( f'(x) = \frac{f(x + h) - f(x - h)}{2h} - \frac{h^2}{6}f'''(\xi) \)\( O(h^2) \)
Second central \( \frac{f(x + h) - 2f(x) + f(x - h)}{h^2} \)\( f''(x) \)\( f''(x) = \frac{f(x + h) - 2f(x) + f(x - h)}{h^2} - \frac{h^2}{12}f^{(4)}(\xi) \)\( O(h^2) \)
Three-point forward \( \frac{-3f(x) + 4f(x + h) - f(x + 2h)}{2h} \)\( f'(x) \)error \( \frac{h^2}{3}f'''(\xi) \)\( O(h^2) \)

Why the central difference wins. In \( f(x \pm h) = f \pm hf' + \frac{h^2}{2}f'' \pm \frac{h^3}{6}f''' + \cdots \), the \( f'' \) terms cancel on subtraction. Similarly the odd terms cancel in the second-difference formula.

For \( f = e^x \), \( x = 0 \), \( h = 0.1 \): forward gives 1.05171 (error \( \approx 0.052 \approx \frac{h}{2} \)), central gives 1.001668 (error \( \approx 0.0017 \approx \frac{h^2}{6} \)), and the second difference gives 1.000834 for \( f''(0) = 1 \) (error \( \approx \frac{h^2}{12} \)).

Derivatives from a difference table

From \( E = e^{hD} \), \( hD = \log(1 + \Delta) = -\log(1 - \nabla) \):

$$hf'(x_0) = \Delta f_0 - \frac{\Delta^2 f_0}{2} + \frac{\Delta^3 f_0}{3} - \frac{\Delta^4 f_0}{4} + \cdots, \qquad h^2 f''(x_0) = \Delta^2 f_0 - \Delta^3 f_0 + \frac{11}{12}\Delta^4 f_0 - \cdots$$$$hf'(x_n) = \nabla f_n + \frac{\nabla^2 f_n}{2} + \frac{\nabla^3 f_n}{3} + \cdots$$

Use the forward series at the top of a table and the backward series at the bottom. Example: from \( f = 1, 8, 27, 64 \) at \( x = 1, 2, 3, 4 \) (\( \Delta f_0 = 7 \), \( \Delta^2 f_0 = 12 \), \( \Delta^3 f_0 = 6 \)): \( f'(1) \approx 7 - 6 + 2 = 3 \), exact for \( x^3 \).

Truncation versus rounding

If each computed value of \( f \) has an error of at most \( \varepsilon \), the forward-difference error is at most

$$E(h) = \frac{2\varepsilon}{h} + \frac{M_2 h}{2}, \qquad M_2 = \max\lvert f'' \rvert,$$

minimised at \( h^* = 2\sqrt{\varepsilon / M_2} \). For the central difference, \( E(h) = \frac{\varepsilon}{h} + \frac{M_3 h^2}{6} \) is minimised at \( h^* = (3\varepsilon / M_3)^{1/3} \). So the best step for a first derivative is about \( \sqrt{\varepsilon} \) (forward) or \( \varepsilon^{1/3} \) (central), not “as small as possible”. Numerical differentiation is ill-conditioned; numerical integration is not.

Richardson extrapolation

If an approximation satisfies \( A(h) = A + ch^p + O(h^{p+1}) \) (or \( O(h^{p+2}) \) when only even powers appear), then

$$\frac{2^p A(h/2) - A(h)}{2^p - 1}$$

cancels the \( h^p \) term. For the central difference (\( p = 2 \), even powers only): \( \frac{4D(h/2) - D(h)}{3} \) is \( O(h^4) \). With \( e^x \) at 0, \( h = 0.1 \): \( D(0.1) \approx 1.0016675 \), \( D(0.05) \approx 1.0004167 \), and the extrapolated value is \( 0.99999979 \), an error of \( 2 \times 10^{-7} \).

Numerical Integration: Newton-Cotes Rules

Integrate the interpolating polynomial on equally spaced nodes \( x_k = a + kh \).

The basic (simple) rules

RuleNodesFormulaError \( \int - Q \)Degree of precision
Midpoint1 (open)\( 2h\,f(x_1) \) on \( [x_0, x_2] \); or \( h\,f(m) \) on width \( h \)\( +\frac{h^3}{24}f''(\xi) \) (width \( h \))1
Trapezoidal2\( \frac{h}{2}(f_0 + f_1) \)\( -\frac{h^3}{12}f''(\xi) \)1
Simpson’s \( \frac{1}{3} \)3\( \frac{h}{3}(f_0 + 4f_1 + f_2) \)\( -\frac{h^5}{90}f^{(4)}(\xi) \)3
Simpson’s \( \frac{3}{8} \)4\( \frac{3h}{8}(f_0 + 3f_1 + 3f_2 + f_3) \)\( -\frac{3h^5}{80}f^{(4)}(\xi) \)3
Boole’s5\( \frac{2h}{45}(7f_0 + 32f_1 + 12f_2 + 32f_3 + 7f_4) \)\( -\frac{8h^7}{945}f^{(6)}(\xi) \)5
Weddle’s7\( \frac{3h}{10}(f_0 + 5f_1 + f_2 + 6f_3 + f_4 + 5f_5 + f_6) \)(not a pure Newton-Cotes rule)5

Finding a rule’s weights. Demand exactness for \( 1, x, x^2, \dots \) in turn. Example: \( \int_{-1}^{1} f \approx \alpha f(-1) + \beta f(0) + \gamma f(1) \). Exact for \( 1, x, x^2 \): \( \alpha + \beta + \gamma = 2 \), \( -\alpha + \gamma = 0 \), \( \alpha + \gamma = \frac{2}{3} \). So \( \alpha = \gamma = \frac{1}{3} \), \( \beta = \frac{4}{3} \): Simpson’s rule, recovered.

Composite rules

Divide \( [a, b] \) into \( n \) subintervals of width \( h = \frac{b - a}{n} \).

$$\textbf{Composite trapezoidal:}\quad T_n = \frac{h}{2}\Big[f_0 + 2(f_1 + f_2 + \dots + f_{n-1}) + f_n\Big], \qquad \int_a^b f - T_n = -\frac{(b - a)h^2}{12}f''(\xi).$$$$\textbf{Composite Simpson's 1/3 } (n \text{ even}):\quad S_n = \frac{h}{3}\Big[f_0 + 4(f_1 + f_3 + \dots + f_{n-1}) + 2(f_2 + f_4 + \dots + f_{n-2}) + f_n\Big],$$$$\int_a^b f - S_n = -\frac{(b - a)h^4}{180}f^{(4)}(\xi).$$

Composite Simpson’s \( \frac{3}{8} \) needs \( n \) a multiple of 3; error \( -\frac{(b - a)h^4}{80}f^{(4)}(\xi) \).

Composite ruleGlobal errorHalving \( h \) divides the error by about
Trapezoidal\( O(h^2) \)4
Simpson’s \( \frac{1}{3} \) or \( \frac{3}{8} \)\( O(h^4) \)16
Boole’s\( O(h^6) \)64

Signs and convexity

Since \( \int - T = -\frac{h^3}{12}f''(\xi) \) and \( \int - M = +\frac{h^3}{24}f''(\xi) \): if \( f'' > 0 \) (convex) on \( [a, b] \), the trapezoidal rule overestimates and the midpoint rule underestimates, and the midpoint error is about half the trapezoidal error with the opposite sign. Combining them to cancel the leading error,

$$\frac{2M + T}{3} = \text{Simpson's rule on } [a, b].$$

If \( f'' \) changes sign on \( [a, b] \), no such conclusion holds. (For \( \frac{1}{1 + x^2} \) on \( [0, 1] \), \( f'' \) changes sign at \( \frac{1}{\sqrt 3} \), and the trapezoidal value above happens to underestimate.)

Romberg integration

The composite trapezoidal error has an expansion in even powers of \( h \) (Euler-Maclaurin): \( T(h) = I + c_1h^2 + c_2h^4 + \cdots \) for smooth \( f \). Richardson extrapolation applies repeatedly:

$$R_{k,1} = \frac{4T(h/2) - T(h)}{3} \ (\text{this is Simpson's rule}), \qquad R_{k,j} = \frac{4^j R_{k,j-1} - R_{k-1,j-1}}{4^j - 1}.$$

A side effect of the same expansion: for a smooth periodic function integrated over a full period, all the \( c_j \) vanish and the trapezoidal rule converges extremely fast.

Gaussian Quadrature

Newton-Cotes fixes the nodes and chooses only the weights. Choosing both nodes and weights, \( n \) points give \( 2n \) free parameters, enough to integrate polynomials of degree \( 2n - 1 \).

\( n \)Nodes \( t_i \)Weights \( w_i \)Exact up to degree
1021 (midpoint rule)
2\( \pm\frac{1}{\sqrt 3} \)1, 13
3\( 0, \pm\sqrt{\frac{3}{5}} \)\( \frac{8}{9}, \frac{5}{9}, \frac{5}{9} \)5

Change of interval. For \( \int_a^b f(x)\,dx \), substitute \( x = \frac{b - a}{2}t + \frac{a + b}{2} \):

$$\int_a^b f(x)\,dx = \frac{b - a}{2}\int_{-1}^{1} f\Big(\frac{b - a}{2}t + \frac{a + b}{2}\Big)dt.$$

Example: two-point Gauss for \( \int_0^1 e^x\,dx \) gives about 1.717896 against the exact \( e - 1 \approx 1.718282 \), using only two function values. The two-point rule fails first on \( x^4 \): it gives \( 2 \cdot \frac{1}{9} = \frac{2}{9} \) for \( \int_{-1}^1 t^4\,dt = \frac{2}{5} \).

Memory Tricks

Don’t Confuse

ThisvsThat
Local (single-panel) error: Simpson \( O(h^5) \)global (composite) error: Simpson \( O(h^4) \)
Simpson’s \( \frac{1}{3} \): \( n \) evenSimpson’s \( \frac{3}{8} \): \( n \) a multiple of 3
Both Simpson rules: degree of precision 33/8 rule has the larger error constant (\( \frac{3}{80} > \frac{1}{90} \) per panel)
Newton-Cotes: fixed equally spaced nodes, \( n \) points give degree \( n - 1 \) or \( n \)Gauss: optimal nodes, \( n \) points give degree \( 2n - 1 \)
Differentiation: rounding error grows as \( h \to 0 \)integration: rounding stays bounded as \( h \to 0 \)
Trapezoid on convex \( f \): overestimatemidpoint on convex \( f \): underestimate

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.

Level 1: Foundation

  1. State the composite trapezoidal rule and its error term.
  2. What is the degree of precision of Simpson’s \( \frac{3}{8} \) rule?
  3. Write the two-point Gauss-Legendre rule on \( [-1, 1] \).
Answers
  1. \( T_n = \frac{h}{2}[f_0 + 2(f_1 + \dots + f_{n-1}) + f_n] \); \( \int_a^b f - T_n = -\frac{(b - a)h^2}{12}f''(\xi) \).
  2. 3 (the same as Simpson’s \( \frac{1}{3} \)).
  3. \( \int_{-1}^{1} f \approx f\left(-\frac{1}{\sqrt 3}\right) + f\left(\frac{1}{\sqrt 3}\right) \), exact for cubics.

Level 2: Part B Style

Practice Q1. Simpson’s \( \frac{1}{3} \) rule with \( h = 1 \) applied to \( \int_0^2 x^4\,dx \) gives

(a) 6.4 (b) \( \frac{20}{3} \) (c) 8 (d) \( \frac{19}{3} \)

Answer: (b)

\( \frac{1}{3}(0 + 4 \cdot 1 + 16) = \frac{20}{3} \approx 6.667 \). The exact value is \( \frac{32}{5} = 6.4 \), and the error \( 6.4 - \frac{20}{3} = -\frac{4}{15} \) agrees with \( -\frac{h^5}{90}f^{(4)} = -\frac{24}{90} = -\frac{4}{15} \) (here \( f^{(4)} = 24 \) is constant).

Practice Q2. The smallest number of subintervals for which the composite trapezoidal error bound for \( \int_0^1 e^{-x^2}\,dx \) is at most \( 10^{-4} \) is

(a) 21 (b) 29 (c) 41 (d) 58

Answer: (c)

\( \max_{[0,1]}\lvert f'' \rvert = 2 \), so the bound is \( \frac{h^2}{6} \le 10^{-4} \), \( h \le 0.0245 \), \( n \ge 40.8 \): \( n = 41 \).

Practice Q3. The rule \( \int_0^1 f(x)\,dx \approx a f(0) + b f(c) \) is to have the highest possible degree of precision. Then

(a) \( a = \frac{1}{4}, b = \frac{3}{4}, c = \frac{2}{3} \) (b) \( a = \frac{1}{2}, b = \frac{1}{2}, c = 1 \) (c) \( a = \frac{1}{3}, b = \frac{2}{3}, c = \frac{3}{4} \) (d) \( a = 0, b = 1, c = \frac{1}{2} \)

Answer: (a)

Exactness for \( 1, x, x^2 \): \( a + b = 1 \), \( bc = \frac{1}{2} \), \( bc^2 = \frac{1}{3} \). Dividing, \( c = \frac{2}{3} \), so \( b = \frac{3}{4} \), \( a = \frac{1}{4} \). For \( x^3 \): \( bc^3 = \frac{3}{4}\cdot\frac{8}{27} = \frac{2}{9} \ne \frac{1}{4} \), so the degree of precision is 2. Options (b) (trapezoid) and (d) (midpoint) have degree 1.

Practice Q4. The central difference approximation \( D(h) = \frac{f(x + h) - f(x - h)}{2h} \) has error \( c_1h^2 + c_2h^4 + \cdots \). The combination that has error \( O(h^4) \) is

(a) \( 2D(h/2) - D(h) \) (b) \( \frac{4D(h/2) - D(h)}{3} \) (c) \( \frac{D(h) + D(h/2)}{2} \) (d) \( \frac{16D(h/2) - D(h)}{15} \)

Answer: (b)

\( D(h/2) = f' + \frac{c_1h^2}{4} + \cdots \). Then \( 4D(h/2) - D(h) = 3f' + O(h^4) \). Option (a) is the extrapolation for an \( O(h) \) method; option (d) is the next Romberg step, which would cancel \( h^4 \) after \( h^2 \) is gone.

Level 3: Part C Style

Practice Q5. For \( I = \int_0^1 \frac{dx}{1 + x^2} \) with \( h = 0.25 \), the composite trapezoidal value is \( T \approx 0.782794 \) and the composite Simpson value is \( S \approx 0.785392 \). Which statements are true?

(a) \( 4S \) agrees with \( \pi \) to four decimal places (after rounding). (b) Since \( f(x) = \frac{1}{1 + x^2} \) is convex on \( [0, 1] \), \( T \) must overestimate \( I \). (c) \( S = \frac{4T_4 - T_2}{3} \), where \( T_2 \), \( T_4 \) are the trapezoidal values with 2 and 4 subintervals. (d) The composite Simpson error bound \( \frac{(b - a)h^4}{180}\max\lvert f^{(4)} \rvert \) shrinks by a factor of 16 if \( h \) is halved.

Answer: (a), (c) and (d)
  • (a) True. \( 4S \approx 3.141569 \), which rounds to 3.1416, as does \( \pi \approx 3.14159 \).
  • (b) False. \( f''(x) = \frac{6x^2 - 2}{(1 + x^2)^3} \) is negative on \( [0, \frac{1}{\sqrt 3}) \) and positive after, so \( f \) is not convex on \( [0, 1] \). In fact \( T < I \approx 0.785398 \).
  • (c) True. This is the first Romberg step: Richardson extrapolation of the trapezoidal rule gives Simpson’s rule on the finer grid. (Check: \( T_2 = \frac{0.5}{2}(1 + 1.6 + 0.5) = 0.775 \), and \( \frac{4(0.782794) - 0.775}{3} \approx 0.785392 \).)
  • (d) True. The bound is proportional to \( h^4 \).

Practice Q6. Which statements about quadrature rules are true?

(a) Simpson’s \( \frac{1}{3} \) rule and Simpson’s \( \frac{3}{8} \) rule have the same degree of precision. (b) The \( n \)-point Gauss-Legendre rule is exact for all polynomials of degree \( 2n - 1 \) and for no polynomial of degree \( 2n \). (c) The two-point Gauss-Legendre rule integrates \( t^4 \) exactly on \( [-1, 1] \). (d) Boole’s rule has degree of precision 5.

Answer: (a), (b) and (d)
  • (a) True. Both have degree 3.
  • (b) True. Exactness up to degree \( 2n - 1 \) is the defining property. For degree \( 2n \), let \( E(p) = \int_{-1}^1 p - \sum w_i p(t_i) \). If \( p = a t^{2n} + (\text{lower terms}) \) with \( a \ne 0 \), then \( E(p) = a\,E(t^{2n}) \), because the lower terms are integrated exactly. And \( E(t^{2n}) = E(\omega^2) \), where \( \omega(t) = \prod_i (t - t_i) \) is monic of degree \( n \): \( E(\omega^2) = \int_{-1}^1 \omega^2 - 0 > 0 \). So \( E(p) \ne 0 \) for every polynomial of exact degree \( 2n \). (For \( n = 1 \): the midpoint rule gives \( 2c \) for \( \int_{-1}^1 (at^2 + bt + c)\,dt = \frac{2a}{3} + 2c \).)
  • (c) False. It gives \( 2 \cdot \frac{1}{9} = \frac{2}{9} \), not \( \frac{2}{5} \).
  • (d) True. It is the 5-point closed Newton-Cotes rule (\( n = 4 \), even), so precision \( n + 1 = 5 \).

Practice Q7. For \( f(x) = e^x \) at \( x = 0 \) with \( h = 0.1 \), a student obtains the forward difference \( 1.05171 \) and the central difference \( 1.00167 \). Which statements are true?

(a) The forward-difference error is close to \( \frac{h}{2}f''(0) = 0.05 \). (b) The central-difference error is close to \( \frac{h^2}{6}f'''(0) \approx 0.00167 \). (c) Halving \( h \) would reduce the central-difference error by a factor of about 2. (d) If each function value carries rounding error up to \( \varepsilon \), the total central-difference error is minimised for \( h \) of order \( \varepsilon^{1/3} \), not as \( h \to 0 \).

Answer: (a), (b) and (d)
  • (a) True. \( 1.05171 - 1 = 0.0517 \approx 0.05 \) (the next Taylor term \( \frac{h^2}{6} \approx 0.0017 \) accounts for the rest).
  • (b) True. \( 1.00167 - 1 = 0.00167 \approx \frac{0.01}{6} \).
  • (c) False. The error is \( O(h^2) \): halving \( h \) divides it by about 4 (\( D(0.05) - 1 \approx 0.000417 \)).
  • (d) True. Total error \( \approx \frac{\varepsilon}{h} + \frac{M_3h^2}{6} \), minimised at \( h = (3\varepsilon/M_3)^{1/3} \).

Quick Revision Table

FormulaError
Forward \( \frac{f(x+h) - f(x)}{h} \)\( O(h) \): \( -\frac{h}{2}f'' \)
Central \( \frac{f(x+h) - f(x-h)}{2h} \)\( O(h^2) \): \( -\frac{h^2}{6}f''' \)
\( \frac{f(x+h) - 2f(x) + f(x-h)}{h^2} \)\( O(h^2) \): \( -\frac{h^2}{12}f^{(4)} \)
\( hD = \log(1 + \Delta) \)\( hf'_0 = \Delta - \frac{\Delta^2}{2} + \frac{\Delta^3}{3} - \cdots \)
Trapezoid, one panel / composite\( -\frac{h^3}{12}f'' \) / \( -\frac{(b-a)h^2}{12}f'' \)
Simpson 1/3, one panel / composite\( -\frac{h^5}{90}f^{(4)} \) / \( -\frac{(b-a)h^4}{180}f^{(4)} \)
Simpson 3/8, one panel\( -\frac{3h^5}{80}f^{(4)} \)
Midpoint, width \( h \)\( +\frac{h^3}{24}f'' \)
Degrees of precisionmidpoint 1, trapezoid 1, Simpson 3, 3/8 3, Boole 5, Weddle 5, Gauss \( n \)-pt \( 2n - 1 \)
Gauss 2-pt / 3-pt\( \pm\frac{1}{\sqrt 3} \), weights 1 / \( 0, \pm\sqrt{3/5} \), weights \( \frac{8}{9}, \frac{5}{9} \)
Richardson (even powers)\( \frac{4A(h/2) - A(h)}{3} \); Romberg step 1 = Simpson

Key Takeaways

What’s Next