Skip to content

Point Estimation and Confidence Intervals

CSIR NET statistics: unbiasedness, consistency, sufficiency, completeness, Cramér–Rao, Rao–Blackwell, UMVUE, MLE and moments, and confidence intervals.

  • 20 min read
  • Updated
On this page · 15 sections
  1. Prerequisites
  2. The Hook: The German Tank Problem
  3. Properties of Estimators
  4. Sufficiency and Completeness
  5. Rao–Blackwell, Lehmann–Scheffé and UMVUE
  6. The Cramér–Rao Lower Bound
  7. Methods of Estimation
  8. Confidence Intervals
  9. Memory Tricks
  10. Don’t Confuse
  11. Common Mistakes
  12. Practice
  13. Quick Revision Table
  14. Key Takeaways
  15. What’s Next

Prerequisites

The Hook: The German Tank Problem

Properties of Estimators

Let \( X_1, \dots, X_n \) be i.i.d. with density or mass function \( f(x; \theta) \), and \( T = T(X_1, \dots, X_n) \) an estimator of \( \tau(\theta) \).

PropertyDefinitionTypical check
Unbiased\( E_\theta(T) = \tau(\theta) \) for all \( \theta \)compute the mean
Bias\( b(\theta) = E_\theta(T) - \tau(\theta) \)
Mean squared error\( E_\theta(T - \tau)^2 = \operatorname{Var}_\theta(T) + b(\theta)^2 \)
Consistent\( T_n \xrightarrow{p} \tau(\theta) \)sufficient: \( E\,T_n \to \tau \) and \( \operatorname{Var}T_n \to 0 \)
Relative efficiency of \( T_1 \) to \( T_2 \) (both unbiased)\( \frac{\operatorname{Var}T_2}{\operatorname{Var}T_1} \)\( > 1 \) means \( T_1 \) is better
Efficientunbiased and attains the Cramér–Rao bound
UMVUEunbiased, with variance \( \le \) that of every unbiased estimator, for every \( \theta \)Lehmann–Scheffé

Facts that decide Part C options.

  • Unbiasedness is not preserved by non-linear maps: \( S^2 \) is unbiased for \( \sigma^2 \) but \( S \) is biased for \( \sigma \) (by Jensen, \( E\,S < \sigma \)).
  • Consistency is preserved by continuous maps: if \( T_n \xrightarrow{p} \theta \), then \( g(T_n) \xrightarrow{p} g(\theta) \).
  • A consistent estimator can be biased (the MLE \( \frac{1}{n}\sum(X_i - \bar{X})^2 \)), and an unbiased one can be inconsistent (\( X_1 \) alone for \( \mu \)).
  • An unbiased estimator need not exist (no unbiased estimator of \( \frac{1}{p} \) from one Bernoulli trial), and a UMVUE can be silly (for Poisson, the UMVUE of \( e^{-2\lambda} \) from one observation is \( (-1)^X \)).

Sufficiency and Completeness

Sufficient statistic
\( T \) is sufficient for \( \theta \) if the conditional distribution of the sample given \( T \) does not depend on \( \theta \). Once you know \( T \), the rest of the data carries no information about \( \theta \).
Neyman–Fisher factorisation theorem

\( T \) is sufficient for \( \theta \) if and only if the joint density factors as

$$f(x_1, \dots, x_n; \theta) = g\left(T(x); \theta\right)\,h(x),$$

where \( h \) does not involve \( \theta \).

Minimal sufficiency. \( T \) is minimal sufficient if \( T(x) = T(y) \) exactly when \( \frac{f(x; \theta)}{f(y; \theta)} \) is free of \( \theta \).

Completeness. \( T \) is complete if \( E_\theta\,g(T) = 0 \) for all \( \theta \) forces \( g(T) = 0 \) with probability 1. A complete sufficient statistic is minimal sufficient.

Exponential family. If \( f(x; \theta) = h(x)\exp\left(\eta(\theta)t(x) - A(\theta)\right) \) and the range of \( \eta(\theta) \) contains an open interval, then \( T = \sum t(X_i) \) is complete sufficient. (For \( k \) parameters, the same with \( k \) statistics and an open set in \( \mathbb{R}^k \).)

FamilyComplete sufficient statistic
Bernoulli\( (p) \), Binomial\( (m, p) \)\( \sum X_i \)
Poisson\( (\lambda) \)\( \sum X_i \)
Exponential, Gamma with known shape\( \sum X_i \)
\( N(\mu, \sigma^2) \), \( \sigma \) known\( \sum X_i \)
\( N(\mu, \sigma^2) \), both unknown\( \left(\sum X_i, \sum X_i^2\right) \), equivalently \( (\bar{X}, S^2) \)
\( U(0, \theta) \)\( X_{(n)} \) (not an exponential family)
\( U(\theta, \theta + 1) \)\( (X_{(1)}, X_{(n)}) \) is minimal sufficient but not complete
Shifted exponential \( e^{-(x - \theta)} \), \( x > \theta \)\( X_{(1)} \)

Basu’s theorem. A complete sufficient statistic is independent of every ancillary statistic (one whose distribution does not depend on \( \theta \)). Example: for \( N(\mu, \sigma^2) \) with \( \sigma \) known, \( \bar{X} \) is complete sufficient for \( \mu \) and \( S^2 \) is ancillary, so they are independent.

Rao–Blackwell, Lehmann–Scheffé and UMVUE

Rao–Blackwell theorem
If \( U \) is unbiased for \( \tau(\theta) \) and \( T \) is sufficient, then \( \phi(T) = E(U \mid T) \) is a statistic, is unbiased for \( \tau(\theta) \), and \( \operatorname{Var}\phi(T) \le \operatorname{Var}U \).
Lehmann–Scheffé theorem
If \( T \) is complete sufficient and \( \phi(T) \) is unbiased for \( \tau(\theta) \), then \( \phi(T) \) is the unique UMVUE of \( \tau(\theta) \).

So the recipe for a UMVUE: find a complete sufficient \( T \); find any function of \( T \) with the right expectation (guess, or condition a simple unbiased estimator on \( T \)).

ModelTargetUMVUENote
Bernoulli\( (p) \), \( T = \sum X_i \)\( p \)\( \frac{T}{n} \)
\( p(1 - p) \)\( \frac{T(n - T)}{n(n - 1)} \)\( E[T(n-T)] = n(n-1)pq \)
\( p^2 \)\( \frac{T(T - 1)}{n(n - 1)} \)
Poisson\( (\lambda) \), \( T = \sum X_i \)\( \lambda \)\( \bar{X} \)
\( e^{-\lambda} = P(X = 0) \)\( \left(1 - \frac{1}{n}\right)^T \)condition \( \mathbf{1}\{X_1 = 0\} \) on \( T \)
\( N(\mu, \sigma^2) \)\( \mu \), \( \sigma^2 \)\( \bar{X} \), \( S^2 \)
Exponential, rate \( \lambda \), \( T = \sum X_i \)mean \( \frac{1}{\lambda} \)\( \bar{X} \)
rate \( \lambda \)\( \frac{n - 1}{T} \)\( E(1/T) = \frac{\lambda}{n - 1} \)
\( U(0, \theta) \)\( \theta \)\( \frac{n + 1}{n}X_{(n)} \)variance \( \frac{\theta^2}{n(n + 2)} \)

The Cramér–Rao Lower Bound

Fisher information

For one observation, under regularity conditions (support not depending on \( \theta \), differentiation under the integral allowed),

$$I(\theta) = E_\theta\left[\left(\frac{\partial}{\partial\theta}\log f(X; \theta)\right)^2\right] = -E_\theta\left[\frac{\partial^2}{\partial\theta^2}\log f(X; \theta)\right].$$

A sample of \( n \) i.i.d. observations carries \( nI(\theta) \).

Cramér–Rao inequality

Under the same conditions, any unbiased estimator \( T \) of \( \tau(\theta) \) satisfies

$$\operatorname{Var}_\theta(T) \ge \frac{[\tau'(\theta)]^2}{nI(\theta)}.$$

Equality holds iff the score is linear in \( T \): \( \frac{\partial}{\partial\theta}\log L = a(\theta)\left(T - \tau(\theta)\right) \), which happens only in a one-parameter exponential family with \( T \) its natural statistic (up to linear transformation).

Model (one observation)\( I(\theta) \)CRLB for unbiased estimators of \( \theta \)
Bernoulli\( (p) \)\( \frac{1}{p(1-p)} \)\( \frac{p(1-p)}{n} \), attained by \( \bar{X} \)
Poisson\( (\lambda) \)\( \frac{1}{\lambda} \)\( \frac{\lambda}{n} \), attained by \( \bar{X} \)
\( N(\mu, \sigma^2) \), for \( \mu \)\( \frac{1}{\sigma^2} \)\( \frac{\sigma^2}{n} \), attained by \( \bar{X} \)
\( N(\mu, \sigma^2) \), for \( \sigma^2 \)\( \frac{1}{2\sigma^4} \)\( \frac{2\sigma^4}{n} \); \( S^2 \) has \( \frac{2\sigma^4}{n-1} \), not attained
Exponential, mean \( \theta \)\( \frac{1}{\theta^2} \)\( \frac{\theta^2}{n} \), attained by \( \bar{X} \)
Exponential, rate \( \lambda \)\( \frac{1}{\lambda^2} \)\( \frac{\lambda^2}{n} \); UMVUE \( \frac{n-1}{T} \) has variance \( \frac{\lambda^2}{n - 2} \), not attained

A UMVUE need not attain the bound (the \( S^2 \) and \( \frac{n-1}{T} \) rows), so “UMVUE” and “efficient” are different properties: efficient implies UMVUE, not conversely.

Methods of Estimation

Method of moments

Equate the first \( k \) sample moments \( \frac{1}{n}\sum X_i^j \) to the population moments \( E(X^j) \) and solve for the \( k \) parameters. Simple, consistent (by the law of large numbers and continuity), often not efficient, and sometimes outside the parameter space.

Examples: \( U(0, \theta) \): \( E\,X = \frac{\theta}{2} \Rightarrow \hat{\theta} = 2\bar{X} \), unbiased with variance \( \frac{\theta^2}{3n} \). Gamma (shape \( \alpha \), rate \( \lambda \)): \( \hat{\alpha} = \frac{\bar{X}^2}{\hat{\sigma}^2} \), \( \hat{\lambda} = \frac{\bar{X}}{\hat{\sigma}^2} \) with \( \hat{\sigma}^2 = \frac{1}{n}\sum (X_i - \bar{X})^2 \).

Maximum likelihood

The likelihood \( L(\theta) = \prod f(x_i; \theta) \) is the joint density read as a function of \( \theta \). The MLE \( \hat{\theta} \) maximises it (usually via \( \ell = \log L \) and \( \ell'(\theta) = 0 \), checking it is a maximum).

ModelMLE
Bernoulli, Poisson, exponential mean\( \bar{X} \)
Exponential rate\( 1/\bar{X} \)
\( N(\mu, \sigma^2) \)\( \hat{\mu} = \bar{X} \), \( \hat{\sigma}^2 = \frac{1}{n}\sum (X_i - \bar{X})^2 \) (biased)
\( U(0, \theta) \)\( X_{(n)} \) (likelihood \( \theta^{-n} \) is decreasing on \( \theta \ge x_{(n)} \))
\( U(-\theta, \theta) \)\( \max \lvert X_i \rvert \)
\( U(\theta - \frac{1}{2}, \theta + \frac{1}{2}) \)any point of \( [X_{(n)} - \frac{1}{2},\, X_{(1)} + \frac{1}{2}] \): not unique
Shifted exponential \( e^{-(x - \theta)} \), \( x > \theta \)\( X_{(1)} \)
Laplace location \( \frac{1}{2}e^{-\lvert x - \theta \rvert} \)the sample median

Properties of the MLE.

  1. Invariance: the MLE of \( g(\theta) \) is \( g(\hat{\theta}) \), for any function \( g \). (The MLE of \( \sigma \) is \( \hat{\sigma} \); of \( e^{-\lambda} \) for Poisson is \( e^{-\bar{X}} \).)
  2. It is a function of any sufficient statistic (when unique).
  3. Under regularity conditions it is consistent and asymptotically normal: \( \sqrt{n}(\hat{\theta} - \theta) \xrightarrow{d} N\left(0, \frac{1}{I(\theta)}\right) \), so asymptotically efficient.
  4. It may be biased, may not be unique, may not exist, and may not be found from \( \ell' = 0 \) (uniform models).
  5. If an efficient estimator exists, the likelihood equation gives it.

Least squares (minimise \( \sum (x_i - \mu_i(\theta))^2 \)) coincides with maximum likelihood when the errors are normal with constant variance.

Confidence Intervals

Confidence interval
A random interval \( [L(X), U(X)] \) is a \( 100(1 - \alpha)\% \) confidence interval for \( \theta \) if \( P_\theta(L \le \theta \le U) \ge 1 - \alpha \) for every \( \theta \). The probability refers to the random interval before the data are seen. Once computed, a particular interval either contains \( \theta \) or does not.

Pivotal method. Find a function \( Q(X, \theta) \) whose distribution is free of \( \theta \) (a pivot), choose \( a, b \) with \( P(a \le Q \le b) = 1 - \alpha \), and invert.

Notation: \( z_\alpha \), \( t_{\nu,\alpha} \), \( \chi^2_{\nu,\alpha} \) denote upper \( \alpha \) points.

ParameterPivot\( 100(1-\alpha)\% \) interval
\( \mu \), normal, \( \sigma \) known\( \frac{\sqrt{n}(\bar{X} - \mu)}{\sigma} \sim N(0,1) \)\( \bar{X} \pm z_{\alpha/2}\frac{\sigma}{\sqrt{n}} \)
\( \mu \), normal, \( \sigma \) unknown\( \frac{\sqrt{n}(\bar{X} - \mu)}{S} \sim t_{n-1} \)\( \bar{X} \pm t_{n-1,\alpha/2}\frac{S}{\sqrt{n}} \)
\( \sigma^2 \), normal\( \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1} \)\( \left[\frac{(n-1)S^2}{\chi^2_{n-1,\alpha/2}},\, \frac{(n-1)S^2}{\chi^2_{n-1,1-\alpha/2}}\right] \)
\( \mu_1 - \mu_2 \), equal unknown variancespooled \( t_{n_1+n_2-2} \)\( \bar{X}_1 - \bar{X}_2 \pm t\,S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}} \)
\( \sigma_1^2/\sigma_2^2 \)\( \frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} \sim F_{n_1-1,n_2-1} \)\( \left[\frac{S_1^2/S_2^2}{F_{\alpha/2}},\, \frac{S_1^2/S_2^2}{F_{1-\alpha/2}}\right] \)
\( p \), large \( n \)\( \frac{\hat{p} - p}{\sqrt{p(1-p)/n}} \approx N(0,1) \)\( \hat{p} \pm z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \)
Exponential mean \( \theta \)\( \frac{2\sum X_i}{\theta} \sim \chi^2_{2n} \)\( \left[\frac{2\sum X_i}{\chi^2_{2n,\alpha/2}},\, \frac{2\sum X_i}{\chi^2_{2n,1-\alpha/2}}\right] \)
\( U(0, \theta) \)\( \frac{X_{(n)}}{\theta} \), CDF \( t^n \) on \( (0,1) \)\( \left[X_{(n)},\, \frac{X_{(n)}}{\alpha^{1/n}}\right] \) (shortest)
any \( \theta \), large \( n \)\( \sqrt{nI(\hat{\theta})}(\hat{\theta} - \theta) \approx N(0,1) \)\( \hat{\theta} \pm \frac{z_{\alpha/2}}{\sqrt{nI(\hat{\theta})}} \)

Common values: \( z_{0.05} = 1.645 \), \( z_{0.025} = 1.96 \), \( z_{0.005} = 2.576 \).

What moves the width. Width of the \( z \)-interval is \( 2z_{\alpha/2}\frac{\sigma}{\sqrt{n}} \): it grows with the confidence level and \( \sigma \), and shrinks like \( \frac{1}{\sqrt{n}} \). To estimate \( \mu \) within \( \pm E \): \( n \ge \left(\frac{z_{\alpha/2}\sigma}{E}\right)^2 \). For a proportion with no prior guess, use \( p = \frac{1}{2} \): \( n \ge \frac{z_{\alpha/2}^2}{4E^2} \).

Intervals and tests are duals. The \( 100(1 - \alpha)\% \) interval is the set of \( \theta_0 \) that a size-\( \alpha \) two-sided test would not reject. See Hypothesis Testing: MP, UMP and Likelihood Ratio Tests.

Memory Tricks

  • “Sufficient squeezes, complete makes it unique.” Rao–Blackwell improves any unbiased estimator by conditioning on a sufficient statistic; completeness guarantees there is only one unbiased function of it.
  • “Hunt the indicator.” When the support depends on \( \theta \), the MLE and sufficient statistic are order statistics, and Cramér–Rao does not apply.
  • \( \frac{n+1}{n}X_{(n)} \). The uniform UMVUE: stretch the maximum by one expected gap.
  • “Information adds.” \( n \) observations carry \( nI(\theta) \); the bound is \( \frac{\tau'^2}{nI} \).
  • “MLE is invariant, unbiasedness is not.” \( g(\hat{\theta}) \) is always the MLE of \( g(\theta) \); \( g \) of an unbiased estimator is rarely unbiased.

Don’t Confuse

ThisIs not thisThe difference
unbiasedconsistentneither implies the other
UMVUEefficient (attains CRLB)efficient \( \Rightarrow \) UMVUE; not conversely
sufficientcomplete\( (X_{(1)}, X_{(n)}) \) for \( U(\theta, \theta+1) \) is sufficient, not complete
MLE of \( \sigma^2 \): divisor \( n \)UMVUE: divisor \( n - 1 \)MLE is biased, UMVUE is not
95% confidence interval“95% probability \( \theta \) is in this interval”the coverage is a property of the procedure
\( \chi^2_{\nu,\alpha} \) upper pointlower pointthe variance interval puts the upper point in the lower limit

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.

Level 1: Foundation

  1. Write the MSE of an estimator in terms of its variance and bias.
  2. What is the MLE of \( \lambda \) for a Poisson sample?
  3. For i.i.d. Bernoulli\( (p) \), name a sufficient statistic.
Answers
  1. \( \text{MSE} = \operatorname{Var}(T) + [\text{bias}]^2 \).
  2. \( \bar{X} \).
  3. \( \sum X_i \) (complete sufficient).

Level 2: Part B Style

Practice Q1. For a random sample \( X_1, \dots, X_n \) from \( U(0, \theta) \), the maximum likelihood estimator of \( \theta \) is:

(a) \( 2\bar{X} \) (b) \( X_{(n)} \) (c) \( \frac{n+1}{n}X_{(n)} \) (d) \( X_{(1)} + X_{(n)} \)

Answer: (b)

\( L(\theta) = \theta^{-n}\mathbf{1}\{\theta \ge x_{(n)}\} \) is maximised at \( \theta = x_{(n)} \). Option (a) is the moment estimator; (c) is the UMVUE.

Practice Q2. The Cramér–Rao lower bound for the variance of unbiased estimators of \( \lambda \), based on a sample of size \( n \) from Poisson\( (\lambda) \), is:

(a) \( \frac{1}{n\lambda} \) (b) \( \frac{\lambda}{n} \) (c) \( \frac{\lambda^2}{n} \) (d) \( \frac{\sqrt{\lambda}}{n} \)

Answer: (b)

\( \log f = -\lambda + x\log\lambda - \log x! \); \( \frac{\partial^2}{\partial\lambda^2} = -\frac{x}{\lambda^2} \); \( I(\lambda) = \frac{1}{\lambda} \). Bound \( \frac{1}{nI} = \frac{\lambda}{n} \), attained by \( \bar{X} \).

Practice Q3. A population has \( \sigma = 10 \). The smallest sample size that estimates \( \mu \) to within \( \pm 2 \) with 95% confidence (using \( z_{0.025} = 1.96 \)) is:

(a) 25 (b) 49 (c) 96 (d) 97

Answer: (d)

\( n \ge \left(\frac{1.96 \times 10}{2}\right)^2 = 96.04 \), so \( n = 97 \). Rounding down to 96 would give a margin slightly above 2.

Practice Q4. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. Bernoulli\( (p) \) and \( T = \sum X_i \). The UMVUE of \( p(1 - p) \) is:

(a) \( \frac{T}{n}\left(1 - \frac{T}{n}\right) \) (b) \( \frac{T(n - T)}{n(n - 1)} \) (c) \( \frac{T(n - T)}{n^2} \) (d) \( \frac{T(T - 1)}{n(n - 1)} \)

Answer: (b)

\( E[T(n - T)] = n \cdot np - (npq + n^2p^2) = n(n-1)pq \). So \( \frac{T(n-T)}{n(n-1)} \) is unbiased and a function of the complete sufficient \( T \): UMVUE by Lehmann–Scheffé. Options (a) and (c) are the same biased plug-in; (d) is the UMVUE of \( p^2 \).

Level 3: Part C Style

Practice Q5. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. \( U(0, \theta) \), \( \theta > 0 \). Which statements are true?

(a) \( 2\bar{X} \) is unbiased for \( \theta \). (b) \( X_{(n)} \) is unbiased for \( \theta \). (c) \( \operatorname{Var}\left(\frac{n+1}{n}X_{(n)}\right) < \operatorname{Var}(2\bar{X}) \). (d) \( X_{(n)} \) is a consistent estimator of \( \theta \).

Answer: (a), (c) and (d)
  • (a) True. \( E\,\bar{X} = \frac{\theta}{2} \).
  • (b) False. \( E\,X_{(n)} = \frac{n\theta}{n+1} < \theta \).
  • (c) True. \( \frac{\theta^2}{n(n+2)} < \frac{\theta^2}{3n} \iff n + 2 > 3 \iff n > 1 \).
  • (d) True. \( P(X_{(n)} < \theta - \varepsilon) = \left(1 - \frac{\varepsilon}{\theta}\right)^n \to 0 \).

Practice Q6. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. \( N(\mu, \sigma^2) \), both parameters unknown. Which statements are true?

(a) \( (\bar{X}, S^2) \) is complete sufficient for \( (\mu, \sigma^2) \). (b) The MLE of \( \sigma^2 \) is unbiased. (c) \( S \) is an unbiased estimator of \( \sigma \). (d) \( \operatorname{Var}(S^2) \) is strictly greater than the Cramér–Rao bound \( \frac{2\sigma^4}{n} \).

Answer: (a) and (d)
  • (a) True. Two-parameter exponential family with natural statistic \( (\sum X_i, \sum X_i^2) \), a one-to-one function of \( (\bar{X}, S^2) \).
  • (b) False. \( E\left[\frac{1}{n}\sum (X_i - \bar{X})^2\right] = \frac{n-1}{n}\sigma^2 \).
  • (c) False. \( E\,S < \sigma \) (Jensen).
  • (d) True. \( \frac{2\sigma^4}{n - 1} > \frac{2\sigma^4}{n} \). \( S^2 \) is the UMVUE but not efficient.

Practice Q7. A random sample of 16 from a normal population gives \( \bar{x} = 50 \) and \( s = 8 \). Use \( t_{15,0.025} = 2.131 \) and \( z_{0.025} = 1.96 \). Which statements are true?

(a) The 95% confidence interval for \( \mu \) is about \( (45.74, 54.26) \). (b) If \( \sigma = 8 \) were known, the 95% interval would be \( (46.08, 53.92) \), narrower than in (a). (c) The 99% \( t \)-interval from the same data is narrower than the 95% one. (d) There is a 95% probability that \( \mu \) lies in the interval computed in (a).

Answer: (a) and (b)
  • (a) True. \( 50 \pm 2.131 \times \frac{8}{4} = 50 \pm 4.262 \).
  • (b) True. \( 50 \pm 1.96 \times 2 = 50 \pm 3.92 \).
  • (c) False. Higher confidence needs a larger multiplier, so a wider interval.
  • (d) False. \( \mu \) is fixed; the 95% describes how often the procedure covers \( \mu \) over repeated samples. (A Bayesian credible interval is the object that carries such a probability statement; see the Bayesian section.)

Practice Q8. \( X_1, \dots, X_n \) are i.i.d. exponential with rate \( \lambda \), \( n \ge 3 \), and \( T = \sum X_i \). Which statements are true?

(a) The MLE of \( \lambda \) is \( \frac{n}{T} \), and it is biased. (b) \( \frac{n - 1}{T} \) is the UMVUE of \( \lambda \). (c) The MLE of \( P(X_1 > 1) = e^{-\lambda} \) is \( e^{-n/T} \). (d) \( \bar{X} \) attains the Cramér–Rao bound for estimating \( \frac{1}{\lambda} \).

Answer: all four
  • (a) True. \( \ell = n\log\lambda - \lambda T \), \( \hat{\lambda} = \frac{n}{T} \); \( E\left(\frac{1}{T}\right) = \frac{\lambda}{n-1} \), so \( E\,\hat{\lambda} = \frac{n\lambda}{n-1} \ne \lambda \).
  • (b) True. Unbiased and a function of the complete sufficient \( T \).
  • (c) True. Invariance of the MLE.
  • (d) True. \( \operatorname{Var}\,\bar{X} = \frac{1}{n\lambda^2} \); the bound for \( \tau(\lambda) = \frac{1}{\lambda} \) is \( \frac{(1/\lambda^2)^2}{n/\lambda^2} = \frac{1}{n\lambda^2} \). The score is linear in \( \bar{X} \).

A Part C question can have every option correct; test each one rather than expecting a false one.

Quick Revision Table

IdeaFact
MSEvariance + bias\( ^2 \)
Consistency (sufficient)bias \( \to 0 \) and variance \( \to 0 \)
Factorisation\( f = g(T; \theta)h(x) \)
Exponential family\( \sum t(X_i) \) complete sufficient
Rao–Blackwell\( E(U \mid T) \) never worse
Lehmann–Schefféunbiased function of complete sufficient = unique UMVUE
Fisher information\( -E[\partial^2_\theta \log f] \); \( n \) observations give \( nI \)
CRLB\( \frac{\tau'(\theta)^2}{nI(\theta)} \); not for \( U(0, \theta) \)
MLEinvariant; asymptotically \( N(\theta, \frac{1}{nI}) \)
\( U(0, \theta) \)MLE \( X_{(n)} \); UMVUE \( \frac{n+1}{n}X_{(n)} \); MoM \( 2\bar{X} \)
\( \mu \) interval\( \bar{X} \pm z\sigma/\sqrt{n} \) or \( \bar{X} \pm tS/\sqrt{n} \)
\( \sigma^2 \) interval\( \frac{(n-1)S^2}{\chi^2_{\text{upper}}} \) to \( \frac{(n-1)S^2}{\chi^2_{\text{lower}}} \)
Sample size\( n \ge (z\sigma/E)^2 \)

Key Takeaways

What’s Next