Skip to content

Random Variables, Distribution Functions and Moments

CSIR NET probability: random variables, CDF properties, discrete, continuous and mixed laws, transformations, expectation, moments, MGF and PGF.

  • 19 min read
  • Updated
On this page · 14 sections
  1. Prerequisites
  2. The Hook: A Waiting Time With an Atom
  3. Random Variables and Distribution Functions
  4. Functions of a Random Variable
  5. Expectation
  6. Moments
  7. Generating Functions
  8. Memory Tricks
  9. Don’t Confuse
  10. Common Mistakes
  11. Practice
  12. Quick Revision Table
  13. Key Takeaways
  14. What’s Next

Prerequisites

The Hook: A Waiting Time With an Atom

Random Variables and Distribution Functions

A random variable on \( (\Omega, \mathcal{F}, P) \) is a function \( X : \Omega \to \mathbb{R} \) with \( \{X \le x\} \in \mathcal{F} \) for every \( x \) (equivalently \( X^{-1}(B) \in \mathcal{F} \) for every Borel set \( B \)). Sums, products, continuous functions, sups and limits of random variables are random variables.

Its distribution function (CDF) is \( F(x) = P(X \le x) \).

Each property comes from continuity of \( P \): \( \{X \le x + \frac{1}{n}\} \downarrow \{X \le x\} \) gives right-continuity, while \( \{X \le x - \frac{1}{n}\} \uparrow \{X < x\} \) gives the left limit \( F(x-) = P(X < x) \).

ProbabilityIn terms of \( F \)
\( P(X = a) \)\( F(a) - F(a-) \) (the jump at \( a \))
\( P(a < X \le b) \)\( F(b) - F(a) \)
\( P(a \le X \le b) \)\( F(b) - F(a-) \)
\( P(a < X < b) \)\( F(b-) - F(a) \)
\( P(X > a) \)\( 1 - F(a) \)

Since \( F \) is monotone it has at most countably many jumps.

Types of distribution

TypeDescriptionExample
Discrete\( F \) is a pure jump function; \( P(X \in S) = 1 \) for a countable \( S \); pmf \( p(x) = P(X = x) \)binomial, Poisson, geometric
Absolutely continuous\( F(x) = \int_{-\infty}^{x} f(t)\,dt \) for a density \( f \ge 0 \), \( \int f = 1 \)normal, exponential, uniform
Singular continuous\( F \) continuous, \( F' = 0 \) almost everywhere, not constantCantor distribution
Mixedcombination of the above\( \min(X, 2) \) in the hook

Lebesgue decomposition. Every CDF is uniquely \( F = \alpha F_d + \beta F_{ac} + \gamma F_{sc} \) with \( \alpha + \beta + \gamma = 1 \), non-negative weights, and \( F_d, F_{ac}, F_{sc} \) discrete, absolutely continuous and singular CDFs.

A density need not be bounded or continuous: \( f(x) = \frac{1}{2\sqrt{x}} \) on \( (0, 1) \) is a density. A density value is not a probability: \( f(x) > 1 \) is allowed.

Quantiles and the probability integral transform

The quantile function \( F^{-1}(u) = \inf\{x : F(x) \ge u\} \), \( u \in (0, 1) \). A median is any \( m \) with \( P(X \le m) \ge \frac{1}{2} \) and \( P(X \ge m) \ge \frac{1}{2} \).

  • If \( U \sim U(0, 1) \), then \( F^{-1}(U) \) has CDF \( F \), for any \( F \). (This is how simulations generate random variables.)
  • If \( F \) is continuous, then \( F(X) \sim U(0, 1) \). With a jump this fails: \( F(X) \) then has an atom.

Functions of a Random Variable

For \( Y = g(X) \), always start from the CDF: \( F_Y(y) = P(g(X) \le y) \).

Monotone \( g \). If \( X \) has density \( f_X \) and \( g \) is strictly monotone and differentiable with inverse \( h \),

$$\boxed{f_Y(y) = f_X(h(y))\,\lvert h'(y) \rvert}$$

on the range of \( g \).

Non-monotone \( g \). Sum over the branches: \( f_Y(y) = \sum_{x : g(x) = y} \frac{f_X(x)}{\lvert g'(x) \rvert} \). For \( Y = X^2 \), \( y > 0 \):

$$f_Y(y) = \frac{f_X(\sqrt{y}) + f_X(-\sqrt{y})}{2\sqrt{y}}.$$
\( X \)\( Y \)Distribution of \( Y \)
\( U(0, 1) \)\( -\frac{1}{\lambda}\log U \)exponential with rate \( \lambda \)
\( N(0, 1) \)\( X^2 \)\( \chi^2_1 \), density \( \frac{1}{\sqrt{2\pi y}}e^{-y/2} \)
\( N(\mu, \sigma^2) \)\( e^X \)lognormal
\( U(-1, 1) \)\( X^2 \)density \( \frac{1}{2\sqrt{y}} \) on \( (0, 1) \)
\( U(-\frac{\pi}{2}, \frac{\pi}{2}) \)\( \tan X \)standard Cauchy
standard Cauchy\( \frac{1}{X} \)standard Cauchy
exponential, rate \( \lambda \)\( \lfloor X \rfloor \)geometric on \( \{0, 1, \dots\} \) with \( P(\lfloor X \rfloor \ge k) = e^{-\lambda k} \)

Expectation

For discrete \( X \), \( E(X) = \sum x\,p(x) \); with density, \( E(X) = \int x f(x)\,dx \); in general \( E(X) = \int X\,dP \). The expectation exists (is finite) iff \( E\lvert X \rvert < \infty \), i.e. the series or integral converges absolutely.

Law of the unconscious statistician (LOTUS). \( E\,g(X) = \sum g(x)p(x) \) or \( \int g(x) f(x)\,dx \): you do not need the distribution of \( g(X) \).

Tail formulas. For \( X \ge 0 \):

$$E(X) = \int_0^\infty P(X > x)\,dx = \int_0^\infty (1 - F(x))\,dx, \qquad E(X^r) = \int_0^\infty r x^{r-1} P(X > x)\,dx.$$

For \( X \) with values in \( \{0, 1, 2, \dots\} \): \( E(X) = \sum_{n \ge 1} P(X \ge n) \). For a general \( X \), \( E(X) = \int_0^\infty (1 - F(x))\,dx - \int_{-\infty}^{0} F(x)\,dx \) when both are finite.

These formulas work for mixed distributions, which is why they are the fastest route in the hook: \( E\min(X, 2) = \int_0^2 e^{-y}\,dy = 1 - e^{-2} \).

Properties. Linearity (always, no independence needed); monotonicity; \( \lvert E X \rvert \le E\lvert X \rvert \); \( E(XY) = E(X)E(Y) \) if \( X, Y \) are independent with finite means.

Moments

QuantityDefinition
\( r \)-th raw moment\( \mu_r' = E(X^r) \)
\( r \)-th central moment\( \mu_r = E(X - \mu)^r \), \( \mu = E X \)
\( r \)-th absolute moment\( E\lvert X \rvert^r \)
Factorial moment\( E[X(X-1)\cdots(X-r+1)] \)
Variance\( \text{Var}(X) = E(X - \mu)^2 = E(X^2) - \mu^2 \)

Central moments from raw moments use the same binomial formulas as for data (see the descriptive statistics page). Skewness \( \gamma_1 = \mu_3/\sigma^3 \), kurtosis \( \beta_2 = \mu_4/\sigma^4 \).

Existence. If \( E\lvert X \rvert^s < \infty \) then \( E\lvert X \rvert^r < \infty \) for all \( 0 < r < s \), since \( \lvert x \rvert^r \le 1 + \lvert x \rvert^s \). The converse fails. For a density with tails like \( \lvert x \rvert^{-(p+1)} \), moments of order \( r < p \) exist and those of order \( r \ge p \) do not. The \( t \)-distribution with \( k \) degrees of freedom has moments exactly of order \( < k \); the Cauchy (\( t_1 \)) has none of order \( \ge 1 \).

Variance rules. \( \text{Var}(aX + b) = a^2\text{Var}(X) \); \( \text{Var}(X) = 0 \) iff \( X \) is almost surely constant; \( E(X - c)^2 = \text{Var}(X) + (\mu - c)^2 \) is least at \( c = \mu \); \( E\lvert X - c \rvert \) is least at any median.

Generating Functions

FunctionDefinitionGives
PGF (for \( X \in \{0, 1, 2, \dots\} \))\( G(s) = E(s^X) \), \( \lvert s \rvert \le 1 \)\( P(X = k) = \frac{G^{(k)}(0)}{k!} \); \( G'(1) = E X \); \( G''(1) = E[X(X-1)] \)
MGF\( M(t) = E(e^{tX}) \)\( M^{(r)}(0) = E(X^r) \) when \( M \) is finite near 0
Cumulant GF\( K(t) = \log M(t) \)\( K'(0) = \mu \), \( K''(0) = \sigma^2 \), \( K'''(0) = \mu_3 \)
Characteristic function\( \varphi(t) = E(e^{itX}) \)always exists; see the next page

Variance from the PGF: \( \text{Var}(X) = G''(1) + G'(1) - G'(1)^2 \).

Key facts about the MGF.

  1. If \( M(t) \) is finite on an open interval around 0, it determines the distribution and all moments exist, with \( M(t) = \sum \frac{E(X^r)}{r!}t^r \) there.
  2. For independent \( X, Y \): \( M_{X+Y} = M_X M_Y \) (likewise for PGFs and characteristic functions).
  3. \( M_{aX + b}(t) = e^{bt}M_X(at) \).
  4. The MGF may be finite only at \( t = 0 \): Cauchy, and also the lognormal, for which every moment \( E(X^r) = e^{r\mu + r^2\sigma^2/2} \) is finite but \( E(e^{tX}) = \infty \) for every \( t > 0 \). Moments of the lognormal do not determine its distribution.
DistributionMeanVarianceMGF \( M(t) \)
Bernoulli(\( p \))\( p \)\( pq \)\( q + pe^t \)
Binomial(\( n, p \))\( np \)\( npq \)\( (q + pe^t)^n \)
Poisson(\( \lambda \))\( \lambda \)\( \lambda \)\( e^{\lambda(e^t - 1)} \)
Geometric on \( \{1, 2, \dots\} \)\( \frac{1}{p} \)\( \frac{q}{p^2} \)\( \frac{pe^t}{1 - qe^t} \), \( t < -\log q \)
Uniform(\( a, b \))\( \frac{a + b}{2} \)\( \frac{(b - a)^2}{12} \)\( \frac{e^{bt} - e^{at}}{t(b - a)} \)
Exponential, rate \( \lambda \)\( \frac{1}{\lambda} \)\( \frac{1}{\lambda^2} \)\( \frac{\lambda}{\lambda - t} \), \( t < \lambda \)
Gamma(\( \alpha \), rate \( \lambda \))\( \frac{\alpha}{\lambda} \)\( \frac{\alpha}{\lambda^2} \)\( \left(\frac{\lambda}{\lambda - t}\right)^\alpha \), \( t < \lambda \)
Normal(\( \mu, \sigma^2 \))\( \mu \)\( \sigma^2 \)\( e^{\mu t + \sigma^2 t^2/2} \)

Here \( q = 1 - p \). The full catalogue of distributions and their properties belongs to the distributions and inference chapter of Unit 4; this table is the minimum you need for this chapter.

Memoryless property. \( P(X > s + t \mid X > s) = P(X > t) \) for all \( s, t \ge 0 \) characterises the exponential among continuous distributions on \( [0, \infty) \), and the geometric on \( \{0, 1, \dots\} \) (or \( \{1, 2, \dots\} \)) among discrete ones.

graph TD
    A[Given: CDF, pmf or density] --> B{Jumps in F?}
    B -->|Only jumps| C[Discrete: sums over atoms]
    B -->|None, F absolutely continuous| D[Density f = F prime]
    B -->|Both| E[Mixed: split into atoms plus density part, or use tail formula]
    C --> F[E X, moments, PGF]
    D --> F
    E --> F
    F --> G{MGF finite near 0?}
    G -->|Yes| H[All moments exist, MGF fixes the law]
    G -->|No| I[Use the characteristic function]

Memory Tricks

Don’t Confuse

ThisvsThat
\( F \) continuous (no atoms)\( F \) absolutely continuous (has a density)
\( F(a) = P(X \le a) \)\( F(a-) = P(X < a) \)
density value \( f(x) \) (can exceed 1)probability \( P(X = x) \) (zero for a continuous \( X \))
symmetric about 0 \( \Rightarrow \) median 0symmetric about 0 \( \Rightarrow \) mean 0 (only if the mean exists)
MGF \( E e^{tX} \) (may not exist)characteristic function \( E e^{itX} \) (always exists)
\( F^{-1}(U) \sim F \) for every \( F \)\( F(X) \sim U(0, 1) \) only for continuous \( F \)

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C of Mathematical Sciences, one or more options may be correct.

Level 1: Foundation

  1. If \( X \sim U(0, 1) \), find \( E(X^2) \).
  2. Which one-sided continuity does every CDF have?
  3. For \( X \) with values in \( \{0, 1, 2, \dots\} \), express \( E X \) through tail probabilities.
Answers
  1. \( \int_0^1 x^2\,dx = \frac{1}{3} \).
  2. Right-continuity: \( F(x) = \lim_{h \downarrow 0} F(x + h) \).
  3. \( E X = \sum_{n \ge 1} P(X \ge n) \).

Level 2: Part B Style

Practice Q1. \( f(x) = c\,x(1 - x) \) on \( (0, 1) \), zero elsewhere, is a density. Its variance is:

(a) \( \frac{1}{12} \) (b) \( \frac{1}{20} \) (c) \( \frac{1}{6} \) (d) \( \frac{3}{10} \)

Answer: (b)

\( \int_0^1 x(1 - x)\,dx = \frac{1}{6} \), so \( c = 6 \). By symmetry \( E X = \frac{1}{2} \). \( E X^2 = 6\int_0^1 (x^3 - x^4)\,dx = 6\left(\frac{1}{4} - \frac{1}{5}\right) = \frac{3}{10} \). Variance \( = \frac{3}{10} - \frac{1}{4} = \frac{1}{20} \). (a) is the uniform variance; (d) is \( E X^2 \).

Practice Q2. \( X \sim U(-1, 1) \) and \( Y = X^2 \). The density of \( Y \) on \( (0, 1) \) is:

(a) \( \frac{1}{2\sqrt{y}} \) (b) \( \frac{1}{4\sqrt{y}} \) (c) \( \frac{1}{\sqrt{y}} \) (d) \( 1 \)

Answer: (a)

\( F_Y(y) = P(-\sqrt{y} \le X \le \sqrt{y}) = \sqrt{y} \), so \( f_Y(y) = \frac{1}{2\sqrt{y}} \). (b) keeps only one branch. Check: \( E Y = \int_0^1 \frac{y}{2\sqrt{y}}\,dy = \frac{1}{3} = E X^2 \).

Practice Q3. A random variable has MGF \( M(t) = \left(\frac{1}{3} + \frac{2}{3}e^t\right)^5 \). Its variance is:

(a) \( \frac{10}{3} \) (b) \( \frac{5}{9} \) (c) \( \frac{10}{9} \) (d) \( \frac{20}{9} \)

Answer: (c)

This is the binomial MGF with \( n = 5 \), \( p = \frac{2}{3} \). By uniqueness, \( \text{Var} = npq = 5 \cdot \frac{2}{3}\cdot\frac{1}{3} = \frac{10}{9} \). (a) is the mean.

Practice Q4. \( X \) is exponential with mean 1 and \( Y = \min(X, 2) \). Then \( E Y \) is:

(a) 1 (b) \( 1 - e^{-2} \) (c) \( 2e^{-2} \) (d) \( 1 - 3e^{-2} \)

Answer: (b)

\( P(Y > y) = e^{-y} \) for \( 0 \le y < 2 \) and 0 for \( y \ge 2 \), so \( E Y = \int_0^2 e^{-y}\,dy = 1 - e^{-2} \approx 0.865 \). Check by splitting: \( \int_0^2 x e^{-x}\,dx + 2P(X > 2) = (1 - 3e^{-2}) + 2e^{-2} \). (d) is the integral part without the atom at 2.

Level 3: Part C Style

Practice Q5. Let

$$F(x) = \begin{cases} 0, & x < 0, \\ \frac{1 + x}{4}, & 0 \le x < 1, \\ \frac{3}{4}, & 1 \le x < 2, \\ 1, & x \ge 2. \end{cases}$$

Which statements are true?

(a) \( P(X = 1) = \frac{1}{4} \). (b) \( P(0 < X < 1) = \frac{1}{4} \). (c) \( X \) is a discrete random variable. (d) \( E X = \frac{7}{8} \).

Answer: (a), (b) and (d)

First check \( F \) is a CDF: non-decreasing, right-continuous, limits 0 and 1. Jumps: \( \frac{1}{4} \) at 0, \( \frac{3}{4} - \frac{1}{2} = \frac{1}{4} \) at 1, \( \frac{1}{4} \) at 2; density \( \frac{1}{4} \) on \( (0, 1) \).

  • (a) True. \( F(1) - F(1-) = \frac{3}{4} - \frac{1}{2} \).
  • (b) True. \( F(1-) - F(0) = \frac{1}{2} - \frac{1}{4} \).
  • (c) False. It has an absolutely continuous part of mass \( \frac{1}{4} \): mixed.
  • (d) True. \( 0 \cdot \frac{1}{4} + \int_0^1 \frac{x}{4}\,dx + 1 \cdot \frac{1}{4} + 2 \cdot \frac{1}{4} = \frac{1}{8} + \frac{3}{4} = \frac{7}{8} \). Tail formula check: \( \int_0^1 \frac{3 - x}{4}\,dx + \int_1^2 \frac{1}{4}\,dx = \frac{5}{8} + \frac{1}{4} = \frac{7}{8} \).

Practice Q6. Let \( X \) have the standard Cauchy density \( \frac{1}{\pi(1 + x^2)} \). Which statements are true?

(a) \( E\lvert X \rvert = \infty \). (b) The median of \( X \) is 0. (c) \( \frac{1}{X} \) has the same distribution as \( X \). (d) \( \text{Var}(X) = 1 \).

Answer: (a), (b) and (c)
  • (a) True. \( \int_0^\infty \frac{x}{1 + x^2}\,dx \) diverges logarithmically.
  • (b) True. The density is symmetric about 0, so \( F(0) = \frac{1}{2} \).
  • (c) True. With \( Y = \frac{1}{X} \), \( f_Y(y) = f_X\left(\frac{1}{y}\right)\frac{1}{y^2} = \frac{1}{\pi(y^2 + 1)} \).
  • (d) False. No moment of order \( \ge 1 \) exists, so there is no variance.

Practice Q7. \( X \) has density \( 2e^{-2x} \) on \( (0, \infty) \). Which statements are true?

(a) \( P(X > 3 \mid X > 1) = e^{-4} \). (b) \( E(X^3) = \frac{3}{4} \). (c) The median of \( X \) exceeds its mean. (d) The MGF of \( X \) is finite for every real \( t \).

Answer: (a) and (b)

Rate \( \lambda = 2 \), mean \( \frac{1}{2} \).

  • (a) True. Memoryless: \( P(X > 2) = e^{-4} \).
  • (b) True. \( E X^r = \frac{r!}{\lambda^r} \), so \( E X^3 = \frac{6}{8} \).
  • (c) False. Median \( = \frac{\log 2}{2} \approx 0.347 < 0.5 \): right skew pulls the mean above the median.
  • (d) False. \( M(t) = \frac{2}{2 - t} \) only for \( t < 2 \); for \( t \ge 2 \) the integral diverges.

Quick Revision Table

IdeaFact
CDF characterisationnon-decreasing, right-continuous, limits 0 and 1
Atom\( P(X = a) = F(a) - F(a-) \); at most countably many
Typesdiscrete, absolutely continuous, singular, mixed (Lebesgue decomposition)
Monotone transform\( f_Y(y) = f_X(h(y))\lvert h'(y) \rvert \)
Inverse transform\( F^{-1}(U) \sim F \); \( F(X) \sim U(0, 1) \) iff \( F \) continuous
Mean existsiff \( E\lvert X \rvert < \infty \)
Tail formula\( E X = \int_0^\infty P(X > x)\,dx \) for \( X \ge 0 \); \( \sum_{n \ge 1} P(X \ge n) \) on integers
Momentsfinite \( s \)-th moment \( \Rightarrow \) all lower ones
PGF\( G'(1) = E X \), \( \text{Var} = G''(1) + G'(1) - G'(1)^2 \)
MGFfinite near 0 \( \Rightarrow \) determines law; \( M^{(r)}(0) = E X^r \)
No MGFCauchy (no mean), lognormal (all moments)
Memorylessexponential (continuous), geometric (discrete)

Key Takeaways

What’s Next