Skip to content

Picard, Euler, Modified Euler and Runge-Kutta

CSIR NET numerical analysis: Picard iteration, Euler and modified Euler methods, Runge-Kutta of orders 2 and 4, local and global error, stability for IVPs.

  • 20 min read
  • Updated
On this page · 18 sections
  1. Prerequisites
  2. The Hook: Three Answers to One Question
  3. The Initial Value Problem
  4. Picard’s Method of Successive Approximations
  5. One-Step Methods: The Vocabulary
  6. Euler’s Method
  7. Modified Euler Methods (Second Order)
  8. The Classical Fourth-Order Runge-Kutta Method
  9. Systems and Higher-Order Equations
  10. Stability: the Test Equation
  11. Comparison Table
  12. Memory Tricks
  13. Don’t Confuse
  14. Common Mistakes
  15. Practice
  16. Quick Revision Table
  17. Key Takeaways
  18. What’s Next

Prerequisites

The Hook: Three Answers to One Question

The Initial Value Problem

$$y' = f(x, y), \qquad y(x_0) = y_0.$$

Integrating, the IVP is equivalent (for continuous \( f \)) to the integral equation

$$y(x) = y_0 + \int_{x_0}^{x} f(t, y(t))\,dt.$$

A bounded \( \frac{\partial f}{\partial y} \) on \( R \) gives a Lipschitz constant. Without the Lipschitz condition, solutions exist (Peano) but need not be unique: \( y' = 3y^{2/3} \), \( y(0) = 0 \) has both \( y \equiv 0 \) and \( y = x^3 \).

Picard’s Method of Successive Approximations

$$y_0(x) \equiv y_0, \qquad y_{n+1}(x) = y_0 + \int_{x_0}^{x} f\big(t, y_n(t)\big)\,dt.$$

This is the fixed-point iteration for the integral equation. Under the hypotheses of the theorem,

$$\lvert y(x) - y_n(x) \rvert \le \frac{M L^n \lvert x - x_0 \rvert^{n+1}}{(n + 1)!},$$

a bound that goes to 0 faster than any geometric sequence.

More examples worth knowing:

IVPIteratesExact solution
\( y' = y \), \( y(0) = 1 \)\( y_n = 1 + x + \dots + \frac{x^n}{n!} \)\( e^x \)
\( y' = 2x(1 + y) \), \( y(0) = 0 \)\( y_1 = x^2 \), \( y_2 = x^2 + \frac{x^4}{2} \), \( y_3 = x^2 + \frac{x^4}{2} + \frac{x^6}{6} \)\( e^{x^2} - 1 \)
\( y' = x^2 + y^2 \), \( y(0) = 0 \)\( y_1 = \frac{x^3}{3} \), \( y_2 = \frac{x^3}{3} + \frac{x^7}{63} \)not elementary

Strengths and limits. Picard gives a formula, not a table, and proves existence. But each step needs an integral in closed form, which quickly becomes impossible (try \( y' = \sin y \)). In practice it is used for theory and for a few terms of a series; for numbers, we use step-by-step methods.

Taylor series method. Differentiate the ODE to get \( y'', y''', \dots \) at \( x_0 \) and sum \( y(x_0 + h) \approx \sum_{k=0}^{p} \frac{h^k}{k!}y^{(k)}(x_0) \). It has order \( p \) but needs derivatives of \( f \); Runge-Kutta methods get the same order from values of \( f \) only.

One-Step Methods: The Vocabulary

Grid \( x_n = x_0 + nh \); \( y_n \approx y(x_n) \). A one-step method is \( y_{n+1} = y_n + h\,\Phi(x_n, y_n; h) \).

  • Local truncation error \( \tau_{n+1} \): the error made in one step when starting from the exact value, \( y(x_{n+1}) - y(x_n) - h\Phi(x_n, y(x_n); h) \).
  • Order \( p \): \( \tau = O(h^{p+1}) \). Then (for Lipschitz \( \Phi \)) the global error \( y(x_n) - y_n \) at a fixed \( x \) is \( O(h^p) \): about \( \frac{x - x_0}{h} \) steps, each adding \( O(h^{p+1}) \).
  • A method has order \( p \) if its expansion in \( h \) agrees with the Taylor series of the true solution through the \( h^p \) term.

Euler’s Method

Follow the tangent line for one step:

$$\boxed{y_{n+1} = y_n + h\,f(x_n, y_n)}$$

It is the left rectangle rule applied to \( \int_{x_n}^{x_{n+1}} f\,dx \).

  • Local truncation error: \( \frac{h^2}{2}y''(\xi) \), so order 1.
  • Global error bound: if \( \lvert y'' \rvert \le M \) and \( f \) is Lipschitz with constant \( L \), then \( \lvert y(x_n) - y_n \rvert \le \frac{hM}{2L}\left(e^{L(x_n - x_0)} - 1\right) \). Halving \( h \) roughly halves the error.

Backward (implicit) Euler \( y_{n+1} = y_n + hf(x_{n+1}, y_{n+1}) \) is also order 1, but needs an equation solved at each step; its reward is stability (below).

Modified Euler Methods (Second Order)

Heun’s method (Euler predictor, trapezoidal corrector)

$$y^{*}_{n+1} = y_n + hf(x_n, y_n), \qquad y_{n+1} = y_n + \frac{h}{2}\Big[f(x_n, y_n) + f(x_{n+1}, y^{*}_{n+1})\Big].$$

It is the trapezoidal rule with the unknown endpoint value predicted by Euler. Some texts iterate the corrector until two successive values agree; the limit is the solution of the implicit trapezoidal equation \( y_{n+1} = y_n + \frac{h}{2}[f(x_n, y_n) + f(x_{n+1}, y_{n+1})] \), which is also second order.

Midpoint method

$$y_{n+1} = y_n + h\,f\Big(x_n + \frac{h}{2},\ y_n + \frac{h}{2}f(x_n, y_n)\Big).$$

It is the midpoint rule with the midpoint value predicted by a half Euler step.

The general two-stage Runge-Kutta family

$$k_1 = f(x_n, y_n), \quad k_2 = f(x_n + \alpha h,\ y_n + \beta h k_1), \quad y_{n+1} = y_n + h(a k_1 + b k_2).$$

Matching the Taylor series through \( h^2 \) gives the order-2 conditions

$$a + b = 1, \qquad b\alpha = \frac{1}{2}, \qquad b\beta = \frac{1}{2}.$$
Method\( a \)\( b \)\( \alpha = \beta \)
Heun (trapezoidal)\( \frac{1}{2} \)\( \frac{1}{2} \)1
Midpoint01\( \frac{1}{2} \)
Ralston\( \frac{1}{4} \)\( \frac{3}{4} \)\( \frac{2}{3} \)

A one-parameter family: every choice of \( b \ne 0 \) gives a second-order method, and no choice gives order 3 for all \( f \). With \( b = 0 \) the scheme is Euler.

The Classical Fourth-Order Runge-Kutta Method

$$\begin{aligned} k_1 &= h\,f(x_n, y_n), \\ k_2 &= h\,f\big(x_n + \tfrac{h}{2},\ y_n + \tfrac{k_1}{2}\big), \\ k_3 &= h\,f\big(x_n + \tfrac{h}{2},\ y_n + \tfrac{k_2}{2}\big), \\ k_4 &= h\,f\big(x_n + h,\ y_n + k_3\big), \\ y_{n+1} &= y_n + \tfrac{1}{6}\big(k_1 + 2k_2 + 2k_3 + k_4\big). \end{aligned}$$

(Here the \( k_i \) include the factor \( h \); some books put \( h \) outside instead. Watch which convention the question uses.)

  • Four evaluations of \( f \) per step; local error \( O(h^5) \), global \( O(h^4) \): halving \( h \) divides the error by about 16.
  • If \( f = f(x) \) only, then \( k_2 = k_3 \) and the step is \( \frac{h}{6}[f(x_n) + 4f(x_n + \frac{h}{2}) + f(x_n + h)] \): Simpson’s rule on \( [x_n, x_{n+1}] \).
  • For \( y' = \lambda y \), one step multiplies \( y \) by \( 1 + z + \frac{z^2}{2} + \frac{z^3}{6} + \frac{z^4}{24} \), \( z = h\lambda \): the Taylor polynomial of \( e^z \) through \( z^4 \).

Systems and Higher-Order Equations

Every method above works component-wise for a system \( \mathbf{y}' = \mathbf{f}(x, \mathbf{y}) \). A higher-order equation is first reduced to a system: for \( y'' = g(x, y, y') \), set \( u = y \), \( v = y' \):

$$u' = v, \qquad v' = g(x, u, v), \qquad u(x_0) = y_0, \quad v(x_0) = y_0'.$$

In RK4 for a system, compute \( k_1 \) for all components before any \( k_2 \), and so on.

Stability: the Test Equation

Apply a method to \( y' = \lambda y \) with \( \operatorname{Re}\lambda < 0 \) (exact solution decays). A one-step method gives \( y_{n+1} = R(h\lambda)\,y_n \), and the numerical solution decays iff \( \lvert R(h\lambda) \rvert < 1 \). The set of such \( z = h\lambda \) is the region of absolute stability.

Method\( R(z) \)Real stability interval (\( \lambda < 0 \))
Euler\( 1 + z \)\( -2 < z < 0 \), i.e. \( h < \frac{2}{\lvert \lambda \rvert} \)
Heun / midpoint (any 2-stage order 2)\( 1 + z + \frac{z^2}{2} \)\( -2 < z < 0 \)
Classical RK4\( 1 + z + \frac{z^2}{2} + \frac{z^3}{6} + \frac{z^4}{24} \)about \( -2.785 < z < 0 \)
Backward Euler\( \frac{1}{1 - z} \)all \( z < 0 \) (whole left half-plane)
Implicit trapezoidal\( \frac{1 + z/2}{1 - z/2} \)all \( z < 0 \) (whole left half-plane)

Example: for \( y' = -20y \), Euler is stable only for \( h < 0.1 \). With \( h = 0.15 \), \( R = 1 - 3 = -2 \) and the numerical solution oscillates with growing amplitude, although the true solution decays. Problems with widely separated time scales (stiff problems) force tiny steps on explicit methods; implicit methods avoid this.

Comparison Table

MethodType\( f \)-evaluations per stepLocal errorGlobal order
Picardsuccessive approximation (analytic)integrals, not evaluations\( \frac{ML^n h^{n+1}}{(n+1)!} \) after \( n \) iterations(not a stepping method)
Eulerexplicit, 1 stage1\( O(h^2) \)1
Backward Eulerimplicitsolve each step\( O(h^2) \)1
Heun / modified Eulerexplicit, 2 stages2\( O(h^3) \)2
Midpointexplicit, 2 stages2\( O(h^3) \)2
Classical RK4explicit, 4 stages4\( O(h^5) \)4

Memory Tricks

Don’t Confuse

ThisvsThat
Local truncation error \( O(h^{p+1}) \)global error \( O(h^p) \)
Heun: slope at the start and at the predicted end, averagedmidpoint: one slope at the predicted midpoint
Picard iterate \( y_n(x) \): a functionEuler value \( y_n \): a number at \( x_n \)
Explicit methods: conditionally stablebackward Euler, implicit trapezoid: stable for every \( h \) when \( \operatorname{Re}\lambda < 0 \)
RK4 \( k_i \) with the \( h \) inside\( k_i \) as slopes, with \( y_{n+1} = y_n + \frac{h}{6}(\cdots) \)
Order of accuracy (how fast error \( \to 0 \))stability (whether errors stay bounded at a given \( h \))

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.

Level 1: Foundation

  1. Write the Picard iteration for \( y' = f(x, y) \), \( y(x_0) = y_0 \).
  2. State the local and global orders of Euler’s method and of classical RK4.
  3. For \( y' = \lambda y \) with \( \lambda < 0 \), what is the largest step for which Euler’s method is stable?
Answers
  1. \( y_{n+1}(x) = y_0 + \int_{x_0}^x f(t, y_n(t))\,dt \), starting from \( y_0(x) \equiv y_0 \).
  2. Euler: local \( O(h^2) \), global \( O(h) \). RK4: local \( O(h^5) \), global \( O(h^4) \).
  3. \( h < \frac{2}{\lvert \lambda \rvert} \) (from \( \lvert 1 + h\lambda \rvert < 1 \)).

Level 2: Part B Style

Practice Q1. For \( y' = 2x(1 + y) \), \( y(0) = 0 \), the second Picard iterate (starting from \( y_0 \equiv 0 \)) is

(a) \( x^2 \) (b) \( x^2 + \frac{x^4}{2} \) (c) \( x^2 + \frac{x^4}{4} \) (d) \( x^2 + x^4 \)

Answer: (b)

\( y_1 = \int_0^x 2t\,dt = x^2 \). \( y_2 = \int_0^x 2t(1 + t^2)\,dt = x^2 + \frac{x^4}{2} \). These are partial sums of the exact solution \( e^{x^2} - 1 = x^2 + \frac{x^4}{2} + \frac{x^6}{6} + \cdots \).

Practice Q2. Euler’s method with \( h = 0.1 \) for \( y' = y^2 \), \( y(0) = 1 \) gives \( y(0.2) \approx \)

(a) 1.2 (b) 1.21 (c) 1.221 (d) 1.25

Answer: (c)

\( y_1 = 1 + 0.1(1)^2 = 1.1 \); \( y_2 = 1.1 + 0.1(1.21) = 1.221 \). (Exact: \( y = \frac{1}{1 - x} \), \( y(0.2) = 1.25 \).)

Practice Q3. Heun’s method (Euler predictor, trapezoidal corrector) with \( h = 0.2 \) for \( y' = -2xy^2 \), \( y(0) = 1 \) gives \( y(0.2) \approx \)

(a) 0.92 (b) 0.96 (c) 0.9615 (d) 1.0

Answer: (b)

\( f(0, 1) = 0 \), so the predictor is \( y^* = 1 \). \( f(0.2, 1) = -0.4 \). Corrector: \( y_1 = 1 + 0.1(0 - 0.4) = 0.96 \). (Exact: \( \frac{1}{1 + x^2} \), \( y(0.2) \approx 0.9615 \); option (d) is Euler’s value.)

Practice Q4. For \( y' = -20y \), the explicit Euler method produces a decaying numerical solution exactly when the step \( h \) satisfies

(a) \( h < 0.05 \) (b) \( h < 0.1 \) (c) \( h < 0.2 \) (d) every \( h > 0 \)

Answer: (b)

\( y_{n+1} = (1 - 20h)y_n \); decay needs \( \lvert 1 - 20h \rvert < 1 \), i.e. \( 0 < h < 0.1 \). For \( 0.05 < h < 0.1 \) the iterates alternate in sign but still decay; (a) is the condition for monotone decay. Option (d) describes backward Euler.

Level 3: Part C Style

Practice Q5. Consider \( y_{n+1} = y_n + h(ak_1 + bk_2) \) with \( k_1 = f(x_n, y_n) \), \( k_2 = f(x_n + \alpha h, y_n + \beta h k_1) \). Which statements are true?

(a) \( a = b = \frac{1}{2} \), \( \alpha = \beta = 1 \) gives a second-order method. (b) \( a = 0 \), \( b = 1 \), \( \alpha = \beta = \frac{1}{2} \) gives a second-order method. (c) \( a = \frac{1}{3} \), \( b = \frac{2}{3} \), \( \alpha = \beta = \frac{3}{4} \) gives a second-order method. (d) Some choice of \( a, b, \alpha, \beta \) gives third order for every smooth \( f \).

Answer: (a), (b) and (c)

Order-2 conditions: \( a + b = 1 \), \( b\alpha = \frac{1}{2} \), \( b\beta = \frac{1}{2} \).

  • (a) True (Heun): \( 1 \), \( \frac{1}{2} \), \( \frac{1}{2} \).
  • (b) True (midpoint): \( 1 \), \( \frac{1}{2} \), \( \frac{1}{2} \).
  • (c) True: \( \frac{1}{3} + \frac{2}{3} = 1 \), \( \frac{2}{3}\cdot\frac{3}{4} = \frac{1}{2} \).
  • (d) False. With two stages, the \( h^3 \) term of the Taylor expansion contains \( f_y(f_x + f f_y) \) type terms that \( k_2 \) cannot produce; test on \( y' = y \): the step multiplies \( y \) by \( 1 + h + b\beta h^2 = 1 + h + \frac{h^2}{2} \), missing \( \frac{h^3}{6} \) whatever the parameters.

Practice Q6. RK4 (with \( k_i \) including the factor \( h \)) is applied to \( y' = y - x \), \( y(0) = 2 \), \( h = 0.1 \). The exact solution is \( y = e^x + x + 1 \). Which statements are true?

(a) \( k_1 = 0.2 \) and \( k_2 = 0.205 \). (b) \( k_3 = 0.20525 \) and \( k_4 = 0.210525 \). (c) \( y_1 \approx 2.2051708 \), within \( 10^{-7} \) of \( y(0.1) \). (d) Halving \( h \) would reduce the global error at \( x = 1 \) by a factor of about 4.

Answer: (a), (b) and (c)
  • (a) True. \( k_1 = 0.1(2 - 0) = 0.2 \); \( k_2 = 0.1(2.1 - 0.05) = 0.205 \).
  • (b) True. \( k_3 = 0.1(2.1025 - 0.05) = 0.20525 \); \( k_4 = 0.1(2.20525 - 0.1) = 0.210525 \).
  • (c) True. \( y_1 = 2 + \frac{0.2 + 0.41 + 0.4105 + 0.210525}{6} = 2 + \frac{1.231025}{6} \approx 2.2051708 \); exact \( e^{0.1} + 1.1 \approx 2.2051709 \), a difference of about \( 8.5 \times 10^{-8} \).
  • (d) False. RK4 is fourth order: halving \( h \) divides the global error by about \( 2^4 = 16 \).

Practice Q7. For the IVP \( y' = \lambda y \), \( y(0) = 1 \), with real \( \lambda < 0 \), which statements are true?

(a) One step of classical RK4 multiplies \( y \) by \( 1 + h\lambda + \frac{(h\lambda)^2}{2} + \frac{(h\lambda)^3}{6} + \frac{(h\lambda)^4}{24} \). (b) Heun’s method is stable for every \( h\lambda \in (-2, 0) \). (c) Classical RK4 is stable for \( h\lambda = -2.5 \), while Euler’s method is not. (d) The Picard iterates for this IVP are \( y_n(x) = \sum_{k=0}^{n}\frac{(\lambda x)^k}{k!} \).

Answer: all four, (a), (b), (c) and (d)
  • (a) True. Compute the stages: \( k_1 = h\lambda y \), \( k_2 = h\lambda(1 + \frac{z}{2})y \), \( k_3 = h\lambda(1 + \frac{z}{2} + \frac{z^2}{4})y \), \( k_4 = h\lambda(1 + z + \frac{z^2}{2} + \frac{z^3}{4})y \) with \( z = h\lambda \); the weighted sum gives the stated factor.
  • (b) True. \( R(z) = 1 + z + \frac{z^2}{2} = \frac{1}{2}\left((z + 1)^2 + 1\right) \), which lies in \( [\frac{1}{2}, 1) \) for \( -2 < z < 0 \).
  • (c) True. \( R(-2.5) = 1 - 2.5 + 3.125 - 2.6042 + 1.6276 \approx 0.648 \), so \( \lvert R \rvert < 1 \); Euler gives \( \lvert 1 - 2.5 \rvert = 1.5 > 1 \).
  • (d) True. \( y_{n+1} = 1 + \int_0^x \lambda y_n(t)\,dt \) adds the next term of the exponential series.

Quick Revision Table

IdeaFact
Picard-Lindelöfcontinuous + Lipschitz in \( y \) \( \Rightarrow \) unique solution on \( \lvert x - x_0 \rvert \le \min(a, \frac{b}{M}) \)
Picard iteration\( y_{n+1} = y_0 + \int_{x_0}^x f(t, y_n(t))\,dt \); error \( \le \frac{ML^n\lvert x - x_0 \rvert^{n+1}}{(n+1)!} \)
Euler\( y_{n+1} = y_n + hf(x_n, y_n) \); order 1
Heun (modified Euler)predictor \( y^* = y_n + hf_n \); \( y_{n+1} = y_n + \frac{h}{2}[f_n + f(x_{n+1}, y^*)] \); order 2
Midpoint\( y_{n+1} = y_n + hf(x_n + \frac{h}{2}, y_n + \frac{h}{2}f_n) \); order 2
RK2 conditions\( a + b = 1 \), \( b\alpha = b\beta = \frac{1}{2} \)
RK4\( y_{n+1} = y_n + \frac{1}{6}(k_1 + 2k_2 + 2k_3 + k_4) \); order 4; Simpson when \( f = f(x) \)
Order \( p \)local \( O(h^{p+1}) \), global \( O(h^p) \)
Stability (real \( h\lambda \))Euler and RK2: \( (-2, 0) \); RK4: about \( (-2.785, 0) \); backward Euler: all negative

Key Takeaways

What’s Next

Advertisement
Advertisement