Random Variables, Distribution Functions and Moments
CSIR NET probability: random variables, CDF properties, discrete, continuous and mixed laws, transformations, expectation, moments, MGF and PGF.
- 19 min
On this page · 14 sections
Prerequisites
- Sample Space, Independence and Bayes’ Theorem: probability spaces, \( \sigma \)-fields and continuity of \( P \), which give the properties of a distribution function.
- Riemann, Improper and Lebesgue Integration: expectation is a Lebesgue integral, and existence of a mean is absolute convergence of an improper integral.
- Continuity, Differentiability and Bounded Variation: a distribution function is monotone, so it has at most countably many jumps.
- Descriptive Statistics and Exploratory Data Analysis: the data versions of mean, variance, \( \beta_1 \), \( \beta_2 \).
The Hook: A Waiting Time With an Atom
Random Variables and Distribution Functions
A random variable on \( (\Omega, \mathcal{F}, P) \) is a function \( X : \Omega \to \mathbb{R} \) with \( \{X \le x\} \in \mathcal{F} \) for every \( x \) (equivalently \( X^{-1}(B) \in \mathcal{F} \) for every Borel set \( B \)). Sums, products, continuous functions, sups and limits of random variables are random variables.
Its distribution function (CDF) is \( F(x) = P(X \le x) \).
Each property comes from continuity of \( P \): \( \{X \le x + \frac{1}{n}\} \downarrow \{X \le x\} \) gives right-continuity, while \( \{X \le x - \frac{1}{n}\} \uparrow \{X < x\} \) gives the left limit \( F(x-) = P(X < x) \).
| Probability | In terms of \( F \) |
|---|---|
| \( P(X = a) \) | \( F(a) - F(a-) \) (the jump at \( a \)) |
| \( P(a < X \le b) \) | \( F(b) - F(a) \) |
| \( P(a \le X \le b) \) | \( F(b) - F(a-) \) |
| \( P(a < X < b) \) | \( F(b-) - F(a) \) |
| \( P(X > a) \) | \( 1 - F(a) \) |
Since \( F \) is monotone it has at most countably many jumps.
Types of distribution
| Type | Description | Example |
|---|---|---|
| Discrete | \( F \) is a pure jump function; \( P(X \in S) = 1 \) for a countable \( S \); pmf \( p(x) = P(X = x) \) | binomial, Poisson, geometric |
| Absolutely continuous | \( F(x) = \int_{-\infty}^{x} f(t)\,dt \) for a density \( f \ge 0 \), \( \int f = 1 \) | normal, exponential, uniform |
| Singular continuous | \( F \) continuous, \( F' = 0 \) almost everywhere, not constant | Cantor distribution |
| Mixed | combination of the above | \( \min(X, 2) \) in the hook |
Lebesgue decomposition. Every CDF is uniquely \( F = \alpha F_d + \beta F_{ac} + \gamma F_{sc} \) with \( \alpha + \beta + \gamma = 1 \), non-negative weights, and \( F_d, F_{ac}, F_{sc} \) discrete, absolutely continuous and singular CDFs.
A density need not be bounded or continuous: \( f(x) = \frac{1}{2\sqrt{x}} \) on \( (0, 1) \) is a density. A density value is not a probability: \( f(x) > 1 \) is allowed.
Quantiles and the probability integral transform
The quantile function \( F^{-1}(u) = \inf\{x : F(x) \ge u\} \), \( u \in (0, 1) \). A median is any \( m \) with \( P(X \le m) \ge \frac{1}{2} \) and \( P(X \ge m) \ge \frac{1}{2} \).
- If \( U \sim U(0, 1) \), then \( F^{-1}(U) \) has CDF \( F \), for any \( F \). (This is how simulations generate random variables.)
- If \( F \) is continuous, then \( F(X) \sim U(0, 1) \). With a jump this fails: \( F(X) \) then has an atom.
Functions of a Random Variable
For \( Y = g(X) \), always start from the CDF: \( F_Y(y) = P(g(X) \le y) \).
Monotone \( g \). If \( X \) has density \( f_X \) and \( g \) is strictly monotone and differentiable with inverse \( h \),
$$\boxed{f_Y(y) = f_X(h(y))\,\lvert h'(y) \rvert}$$on the range of \( g \).
Non-monotone \( g \). Sum over the branches: \( f_Y(y) = \sum_{x : g(x) = y} \frac{f_X(x)}{\lvert g'(x) \rvert} \). For \( Y = X^2 \), \( y > 0 \):
$$f_Y(y) = \frac{f_X(\sqrt{y}) + f_X(-\sqrt{y})}{2\sqrt{y}}.$$| \( X \) | \( Y \) | Distribution of \( Y \) |
|---|---|---|
| \( U(0, 1) \) | \( -\frac{1}{\lambda}\log U \) | exponential with rate \( \lambda \) |
| \( N(0, 1) \) | \( X^2 \) | \( \chi^2_1 \), density \( \frac{1}{\sqrt{2\pi y}}e^{-y/2} \) |
| \( N(\mu, \sigma^2) \) | \( e^X \) | lognormal |
| \( U(-1, 1) \) | \( X^2 \) | density \( \frac{1}{2\sqrt{y}} \) on \( (0, 1) \) |
| \( U(-\frac{\pi}{2}, \frac{\pi}{2}) \) | \( \tan X \) | standard Cauchy |
| standard Cauchy | \( \frac{1}{X} \) | standard Cauchy |
| exponential, rate \( \lambda \) | \( \lfloor X \rfloor \) | geometric on \( \{0, 1, \dots\} \) with \( P(\lfloor X \rfloor \ge k) = e^{-\lambda k} \) |
Expectation
For discrete \( X \), \( E(X) = \sum x\,p(x) \); with density, \( E(X) = \int x f(x)\,dx \); in general \( E(X) = \int X\,dP \). The expectation exists (is finite) iff \( E\lvert X \rvert < \infty \), i.e. the series or integral converges absolutely.
Law of the unconscious statistician (LOTUS). \( E\,g(X) = \sum g(x)p(x) \) or \( \int g(x) f(x)\,dx \): you do not need the distribution of \( g(X) \).
Tail formulas. For \( X \ge 0 \):
$$E(X) = \int_0^\infty P(X > x)\,dx = \int_0^\infty (1 - F(x))\,dx, \qquad E(X^r) = \int_0^\infty r x^{r-1} P(X > x)\,dx.$$For \( X \) with values in \( \{0, 1, 2, \dots\} \): \( E(X) = \sum_{n \ge 1} P(X \ge n) \). For a general \( X \), \( E(X) = \int_0^\infty (1 - F(x))\,dx - \int_{-\infty}^{0} F(x)\,dx \) when both are finite.
These formulas work for mixed distributions, which is why they are the fastest route in the hook: \( E\min(X, 2) = \int_0^2 e^{-y}\,dy = 1 - e^{-2} \).
Properties. Linearity (always, no independence needed); monotonicity; \( \lvert E X \rvert \le E\lvert X \rvert \); \( E(XY) = E(X)E(Y) \) if \( X, Y \) are independent with finite means.
Moments
| Quantity | Definition |
|---|---|
| \( r \)-th raw moment | \( \mu_r' = E(X^r) \) |
| \( r \)-th central moment | \( \mu_r = E(X - \mu)^r \), \( \mu = E X \) |
| \( r \)-th absolute moment | \( E\lvert X \rvert^r \) |
| Factorial moment | \( E[X(X-1)\cdots(X-r+1)] \) |
| Variance | \( \text{Var}(X) = E(X - \mu)^2 = E(X^2) - \mu^2 \) |
Central moments from raw moments use the same binomial formulas as for data (see the descriptive statistics page). Skewness \( \gamma_1 = \mu_3/\sigma^3 \), kurtosis \( \beta_2 = \mu_4/\sigma^4 \).
Existence. If \( E\lvert X \rvert^s < \infty \) then \( E\lvert X \rvert^r < \infty \) for all \( 0 < r < s \), since \( \lvert x \rvert^r \le 1 + \lvert x \rvert^s \). The converse fails. For a density with tails like \( \lvert x \rvert^{-(p+1)} \), moments of order \( r < p \) exist and those of order \( r \ge p \) do not. The \( t \)-distribution with \( k \) degrees of freedom has moments exactly of order \( < k \); the Cauchy (\( t_1 \)) has none of order \( \ge 1 \).
Variance rules. \( \text{Var}(aX + b) = a^2\text{Var}(X) \); \( \text{Var}(X) = 0 \) iff \( X \) is almost surely constant; \( E(X - c)^2 = \text{Var}(X) + (\mu - c)^2 \) is least at \( c = \mu \); \( E\lvert X - c \rvert \) is least at any median.
Generating Functions
| Function | Definition | Gives |
|---|---|---|
| PGF (for \( X \in \{0, 1, 2, \dots\} \)) | \( G(s) = E(s^X) \), \( \lvert s \rvert \le 1 \) | \( P(X = k) = \frac{G^{(k)}(0)}{k!} \); \( G'(1) = E X \); \( G''(1) = E[X(X-1)] \) |
| MGF | \( M(t) = E(e^{tX}) \) | \( M^{(r)}(0) = E(X^r) \) when \( M \) is finite near 0 |
| Cumulant GF | \( K(t) = \log M(t) \) | \( K'(0) = \mu \), \( K''(0) = \sigma^2 \), \( K'''(0) = \mu_3 \) |
| Characteristic function | \( \varphi(t) = E(e^{itX}) \) | always exists; see the next page |
Variance from the PGF: \( \text{Var}(X) = G''(1) + G'(1) - G'(1)^2 \).
Key facts about the MGF.
- If \( M(t) \) is finite on an open interval around 0, it determines the distribution and all moments exist, with \( M(t) = \sum \frac{E(X^r)}{r!}t^r \) there.
- For independent \( X, Y \): \( M_{X+Y} = M_X M_Y \) (likewise for PGFs and characteristic functions).
- \( M_{aX + b}(t) = e^{bt}M_X(at) \).
- The MGF may be finite only at \( t = 0 \): Cauchy, and also the lognormal, for which every moment \( E(X^r) = e^{r\mu + r^2\sigma^2/2} \) is finite but \( E(e^{tX}) = \infty \) for every \( t > 0 \). Moments of the lognormal do not determine its distribution.
| Distribution | Mean | Variance | MGF \( M(t) \) |
|---|---|---|---|
| Bernoulli(\( p \)) | \( p \) | \( pq \) | \( q + pe^t \) |
| Binomial(\( n, p \)) | \( np \) | \( npq \) | \( (q + pe^t)^n \) |
| Poisson(\( \lambda \)) | \( \lambda \) | \( \lambda \) | \( e^{\lambda(e^t - 1)} \) |
| Geometric on \( \{1, 2, \dots\} \) | \( \frac{1}{p} \) | \( \frac{q}{p^2} \) | \( \frac{pe^t}{1 - qe^t} \), \( t < -\log q \) |
| Uniform(\( a, b \)) | \( \frac{a + b}{2} \) | \( \frac{(b - a)^2}{12} \) | \( \frac{e^{bt} - e^{at}}{t(b - a)} \) |
| Exponential, rate \( \lambda \) | \( \frac{1}{\lambda} \) | \( \frac{1}{\lambda^2} \) | \( \frac{\lambda}{\lambda - t} \), \( t < \lambda \) |
| Gamma(\( \alpha \), rate \( \lambda \)) | \( \frac{\alpha}{\lambda} \) | \( \frac{\alpha}{\lambda^2} \) | \( \left(\frac{\lambda}{\lambda - t}\right)^\alpha \), \( t < \lambda \) |
| Normal(\( \mu, \sigma^2 \)) | \( \mu \) | \( \sigma^2 \) | \( e^{\mu t + \sigma^2 t^2/2} \) |
Here \( q = 1 - p \). The full catalogue of distributions and their properties belongs to the distributions and inference chapter of Unit 4; this table is the minimum you need for this chapter.
Memoryless property. \( P(X > s + t \mid X > s) = P(X > t) \) for all \( s, t \ge 0 \) characterises the exponential among continuous distributions on \( [0, \infty) \), and the geometric on \( \{0, 1, \dots\} \) (or \( \{1, 2, \dots\} \)) among discrete ones.
graph TD
A[Given: CDF, pmf or density] --> B{Jumps in F?}
B -->|Only jumps| C[Discrete: sums over atoms]
B -->|None, F absolutely continuous| D[Density f = F prime]
B -->|Both| E[Mixed: split into atoms plus density part, or use tail formula]
C --> F[E X, moments, PGF]
D --> F
E --> F
F --> G{MGF finite near 0?}
G -->|Yes| H[All moments exist, MGF fixes the law]
G -->|No| I[Use the characteristic function]Memory Tricks
Don’t Confuse
| This | vs | That |
|---|---|---|
| \( F \) continuous (no atoms) | \( F \) absolutely continuous (has a density) | |
| \( F(a) = P(X \le a) \) | \( F(a-) = P(X < a) \) | |
| density value \( f(x) \) (can exceed 1) | probability \( P(X = x) \) (zero for a continuous \( X \)) | |
| symmetric about 0 \( \Rightarrow \) median 0 | symmetric about 0 \( \Rightarrow \) mean 0 (only if the mean exists) | |
| MGF \( E e^{tX} \) (may not exist) | characteristic function \( E e^{itX} \) (always exists) | |
| \( F^{-1}(U) \sim F \) for every \( F \) | \( F(X) \sim U(0, 1) \) only for continuous \( F \) |
Common Mistakes
Practice
All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C of Mathematical Sciences, one or more options may be correct.
Level 1: Foundation
- If \( X \sim U(0, 1) \), find \( E(X^2) \).
- Which one-sided continuity does every CDF have?
- For \( X \) with values in \( \{0, 1, 2, \dots\} \), express \( E X \) through tail probabilities.
Answers
- \( \int_0^1 x^2\,dx = \frac{1}{3} \).
- Right-continuity: \( F(x) = \lim_{h \downarrow 0} F(x + h) \).
- \( E X = \sum_{n \ge 1} P(X \ge n) \).
Level 2: Part B Style
Practice Q1. \( f(x) = c\,x(1 - x) \) on \( (0, 1) \), zero elsewhere, is a density. Its variance is:
(a) \( \frac{1}{12} \) (b) \( \frac{1}{20} \) (c) \( \frac{1}{6} \) (d) \( \frac{3}{10} \)
Answer: (b)
\( \int_0^1 x(1 - x)\,dx = \frac{1}{6} \), so \( c = 6 \). By symmetry \( E X = \frac{1}{2} \). \( E X^2 = 6\int_0^1 (x^3 - x^4)\,dx = 6\left(\frac{1}{4} - \frac{1}{5}\right) = \frac{3}{10} \). Variance \( = \frac{3}{10} - \frac{1}{4} = \frac{1}{20} \). (a) is the uniform variance; (d) is \( E X^2 \).
Practice Q2. \( X \sim U(-1, 1) \) and \( Y = X^2 \). The density of \( Y \) on \( (0, 1) \) is:
(a) \( \frac{1}{2\sqrt{y}} \) (b) \( \frac{1}{4\sqrt{y}} \) (c) \( \frac{1}{\sqrt{y}} \) (d) \( 1 \)
Answer: (a)
\( F_Y(y) = P(-\sqrt{y} \le X \le \sqrt{y}) = \sqrt{y} \), so \( f_Y(y) = \frac{1}{2\sqrt{y}} \). (b) keeps only one branch. Check: \( E Y = \int_0^1 \frac{y}{2\sqrt{y}}\,dy = \frac{1}{3} = E X^2 \).
Practice Q3. A random variable has MGF \( M(t) = \left(\frac{1}{3} + \frac{2}{3}e^t\right)^5 \). Its variance is:
(a) \( \frac{10}{3} \) (b) \( \frac{5}{9} \) (c) \( \frac{10}{9} \) (d) \( \frac{20}{9} \)
Answer: (c)
This is the binomial MGF with \( n = 5 \), \( p = \frac{2}{3} \). By uniqueness, \( \text{Var} = npq = 5 \cdot \frac{2}{3}\cdot\frac{1}{3} = \frac{10}{9} \). (a) is the mean.
Practice Q4. \( X \) is exponential with mean 1 and \( Y = \min(X, 2) \). Then \( E Y \) is:
(a) 1 (b) \( 1 - e^{-2} \) (c) \( 2e^{-2} \) (d) \( 1 - 3e^{-2} \)
Answer: (b)
\( P(Y > y) = e^{-y} \) for \( 0 \le y < 2 \) and 0 for \( y \ge 2 \), so \( E Y = \int_0^2 e^{-y}\,dy = 1 - e^{-2} \approx 0.865 \). Check by splitting: \( \int_0^2 x e^{-x}\,dx + 2P(X > 2) = (1 - 3e^{-2}) + 2e^{-2} \). (d) is the integral part without the atom at 2.
Level 3: Part C Style
Practice Q5. Let
$$F(x) = \begin{cases} 0, & x < 0, \\ \frac{1 + x}{4}, & 0 \le x < 1, \\ \frac{3}{4}, & 1 \le x < 2, \\ 1, & x \ge 2. \end{cases}$$Which statements are true?
(a) \( P(X = 1) = \frac{1}{4} \). (b) \( P(0 < X < 1) = \frac{1}{4} \). (c) \( X \) is a discrete random variable. (d) \( E X = \frac{7}{8} \).
Answer: (a), (b) and (d)
First check \( F \) is a CDF: non-decreasing, right-continuous, limits 0 and 1. Jumps: \( \frac{1}{4} \) at 0, \( \frac{3}{4} - \frac{1}{2} = \frac{1}{4} \) at 1, \( \frac{1}{4} \) at 2; density \( \frac{1}{4} \) on \( (0, 1) \).
- (a) True. \( F(1) - F(1-) = \frac{3}{4} - \frac{1}{2} \).
- (b) True. \( F(1-) - F(0) = \frac{1}{2} - \frac{1}{4} \).
- (c) False. It has an absolutely continuous part of mass \( \frac{1}{4} \): mixed.
- (d) True. \( 0 \cdot \frac{1}{4} + \int_0^1 \frac{x}{4}\,dx + 1 \cdot \frac{1}{4} + 2 \cdot \frac{1}{4} = \frac{1}{8} + \frac{3}{4} = \frac{7}{8} \). Tail formula check: \( \int_0^1 \frac{3 - x}{4}\,dx + \int_1^2 \frac{1}{4}\,dx = \frac{5}{8} + \frac{1}{4} = \frac{7}{8} \).
Practice Q6. Let \( X \) have the standard Cauchy density \( \frac{1}{\pi(1 + x^2)} \). Which statements are true?
(a) \( E\lvert X \rvert = \infty \). (b) The median of \( X \) is 0. (c) \( \frac{1}{X} \) has the same distribution as \( X \). (d) \( \text{Var}(X) = 1 \).
Answer: (a), (b) and (c)
- (a) True. \( \int_0^\infty \frac{x}{1 + x^2}\,dx \) diverges logarithmically.
- (b) True. The density is symmetric about 0, so \( F(0) = \frac{1}{2} \).
- (c) True. With \( Y = \frac{1}{X} \), \( f_Y(y) = f_X\left(\frac{1}{y}\right)\frac{1}{y^2} = \frac{1}{\pi(y^2 + 1)} \).
- (d) False. No moment of order \( \ge 1 \) exists, so there is no variance.
Practice Q7. \( X \) has density \( 2e^{-2x} \) on \( (0, \infty) \). Which statements are true?
(a) \( P(X > 3 \mid X > 1) = e^{-4} \). (b) \( E(X^3) = \frac{3}{4} \). (c) The median of \( X \) exceeds its mean. (d) The MGF of \( X \) is finite for every real \( t \).
Answer: (a) and (b)
Rate \( \lambda = 2 \), mean \( \frac{1}{2} \).
- (a) True. Memoryless: \( P(X > 2) = e^{-4} \).
- (b) True. \( E X^r = \frac{r!}{\lambda^r} \), so \( E X^3 = \frac{6}{8} \).
- (c) False. Median \( = \frac{\log 2}{2} \approx 0.347 < 0.5 \): right skew pulls the mean above the median.
- (d) False. \( M(t) = \frac{2}{2 - t} \) only for \( t < 2 \); for \( t \ge 2 \) the integral diverges.
Quick Revision Table
| Idea | Fact |
|---|---|
| CDF characterisation | non-decreasing, right-continuous, limits 0 and 1 |
| Atom | \( P(X = a) = F(a) - F(a-) \); at most countably many |
| Types | discrete, absolutely continuous, singular, mixed (Lebesgue decomposition) |
| Monotone transform | \( f_Y(y) = f_X(h(y))\lvert h'(y) \rvert \) |
| Inverse transform | \( F^{-1}(U) \sim F \); \( F(X) \sim U(0, 1) \) iff \( F \) continuous |
| Mean exists | iff \( E\lvert X \rvert < \infty \) |
| Tail formula | \( E X = \int_0^\infty P(X > x)\,dx \) for \( X \ge 0 \); \( \sum_{n \ge 1} P(X \ge n) \) on integers |
| Moments | finite \( s \)-th moment \( \Rightarrow \) all lower ones |
| PGF | \( G'(1) = E X \), \( \text{Var} = G''(1) + G'(1) - G'(1)^2 \) |
| MGF | finite near 0 \( \Rightarrow \) determines law; \( M^{(r)}(0) = E X^r \) |
| No MGF | Cauchy (no mean), lognormal (all moments) |
| Memoryless | exponential (continuous), geometric (discrete) |
Key Takeaways
What’s Next
- Joint, Marginal and Conditional Distributions; Characteristic Functions: several random variables at once, conditional expectation, and the transform that always exists.
- Inequalities, Convergence and Limit Theorems: what moments buy you (Markov, Chebyshev, Jensen) and where sums of random variables go.
- Sequences and Series of Functions for the power-series facts behind MGFs and PGFs (radius of convergence, termwise differentiation).
- Probability Theory: Quick Revision.