Point Estimation and Confidence Intervals
CSIR NET statistics: unbiasedness, consistency, sufficiency, completeness, Cramér–Rao, Rao–Blackwell, UMVUE, MLE and moments, and confidence intervals.
- 20 min
On this page · 15 sections
Prerequisites
- Standard Discrete and Continuous Distributions: the families whose parameters we estimate.
- Sampling Distributions, Standard Errors and Order Statistics: \( \chi^2 \), \( t \), \( F \) and the distribution of \( X_{(n)} \), used as pivots.
- Continuity and Differentiability: maximising a likelihood is a calculus problem, except when the maximum sits at a corner where the derivative is useless.
The Hook: The German Tank Problem
Properties of Estimators
Let \( X_1, \dots, X_n \) be i.i.d. with density or mass function \( f(x; \theta) \), and \( T = T(X_1, \dots, X_n) \) an estimator of \( \tau(\theta) \).
| Property | Definition | Typical check |
|---|---|---|
| Unbiased | \( E_\theta(T) = \tau(\theta) \) for all \( \theta \) | compute the mean |
| Bias | \( b(\theta) = E_\theta(T) - \tau(\theta) \) | |
| Mean squared error | \( E_\theta(T - \tau)^2 = \operatorname{Var}_\theta(T) + b(\theta)^2 \) | |
| Consistent | \( T_n \xrightarrow{p} \tau(\theta) \) | sufficient: \( E\,T_n \to \tau \) and \( \operatorname{Var}T_n \to 0 \) |
| Relative efficiency of \( T_1 \) to \( T_2 \) (both unbiased) | \( \frac{\operatorname{Var}T_2}{\operatorname{Var}T_1} \) | \( > 1 \) means \( T_1 \) is better |
| Efficient | unbiased and attains the Cramér–Rao bound | |
| UMVUE | unbiased, with variance \( \le \) that of every unbiased estimator, for every \( \theta \) | Lehmann–Scheffé |
Facts that decide Part C options.
- Unbiasedness is not preserved by non-linear maps: \( S^2 \) is unbiased for \( \sigma^2 \) but \( S \) is biased for \( \sigma \) (by Jensen, \( E\,S < \sigma \)).
- Consistency is preserved by continuous maps: if \( T_n \xrightarrow{p} \theta \), then \( g(T_n) \xrightarrow{p} g(\theta) \).
- A consistent estimator can be biased (the MLE \( \frac{1}{n}\sum(X_i - \bar{X})^2 \)), and an unbiased one can be inconsistent (\( X_1 \) alone for \( \mu \)).
- An unbiased estimator need not exist (no unbiased estimator of \( \frac{1}{p} \) from one Bernoulli trial), and a UMVUE can be silly (for Poisson, the UMVUE of \( e^{-2\lambda} \) from one observation is \( (-1)^X \)).
Sufficiency and Completeness
\( T \) is sufficient for \( \theta \) if and only if the joint density factors as
$$f(x_1, \dots, x_n; \theta) = g\left(T(x); \theta\right)\,h(x),$$where \( h \) does not involve \( \theta \).
Minimal sufficiency. \( T \) is minimal sufficient if \( T(x) = T(y) \) exactly when \( \frac{f(x; \theta)}{f(y; \theta)} \) is free of \( \theta \).
Completeness. \( T \) is complete if \( E_\theta\,g(T) = 0 \) for all \( \theta \) forces \( g(T) = 0 \) with probability 1. A complete sufficient statistic is minimal sufficient.
Exponential family. If \( f(x; \theta) = h(x)\exp\left(\eta(\theta)t(x) - A(\theta)\right) \) and the range of \( \eta(\theta) \) contains an open interval, then \( T = \sum t(X_i) \) is complete sufficient. (For \( k \) parameters, the same with \( k \) statistics and an open set in \( \mathbb{R}^k \).)
| Family | Complete sufficient statistic |
|---|---|
| Bernoulli\( (p) \), Binomial\( (m, p) \) | \( \sum X_i \) |
| Poisson\( (\lambda) \) | \( \sum X_i \) |
| Exponential, Gamma with known shape | \( \sum X_i \) |
| \( N(\mu, \sigma^2) \), \( \sigma \) known | \( \sum X_i \) |
| \( N(\mu, \sigma^2) \), both unknown | \( \left(\sum X_i, \sum X_i^2\right) \), equivalently \( (\bar{X}, S^2) \) |
| \( U(0, \theta) \) | \( X_{(n)} \) (not an exponential family) |
| \( U(\theta, \theta + 1) \) | \( (X_{(1)}, X_{(n)}) \) is minimal sufficient but not complete |
| Shifted exponential \( e^{-(x - \theta)} \), \( x > \theta \) | \( X_{(1)} \) |
Basu’s theorem. A complete sufficient statistic is independent of every ancillary statistic (one whose distribution does not depend on \( \theta \)). Example: for \( N(\mu, \sigma^2) \) with \( \sigma \) known, \( \bar{X} \) is complete sufficient for \( \mu \) and \( S^2 \) is ancillary, so they are independent.
Rao–Blackwell, Lehmann–Scheffé and UMVUE
So the recipe for a UMVUE: find a complete sufficient \( T \); find any function of \( T \) with the right expectation (guess, or condition a simple unbiased estimator on \( T \)).
| Model | Target | UMVUE | Note |
|---|---|---|---|
| Bernoulli\( (p) \), \( T = \sum X_i \) | \( p \) | \( \frac{T}{n} \) | |
| \( p(1 - p) \) | \( \frac{T(n - T)}{n(n - 1)} \) | \( E[T(n-T)] = n(n-1)pq \) | |
| \( p^2 \) | \( \frac{T(T - 1)}{n(n - 1)} \) | ||
| Poisson\( (\lambda) \), \( T = \sum X_i \) | \( \lambda \) | \( \bar{X} \) | |
| \( e^{-\lambda} = P(X = 0) \) | \( \left(1 - \frac{1}{n}\right)^T \) | condition \( \mathbf{1}\{X_1 = 0\} \) on \( T \) | |
| \( N(\mu, \sigma^2) \) | \( \mu \), \( \sigma^2 \) | \( \bar{X} \), \( S^2 \) | |
| Exponential, rate \( \lambda \), \( T = \sum X_i \) | mean \( \frac{1}{\lambda} \) | \( \bar{X} \) | |
| rate \( \lambda \) | \( \frac{n - 1}{T} \) | \( E(1/T) = \frac{\lambda}{n - 1} \) | |
| \( U(0, \theta) \) | \( \theta \) | \( \frac{n + 1}{n}X_{(n)} \) | variance \( \frac{\theta^2}{n(n + 2)} \) |
The Cramér–Rao Lower Bound
For one observation, under regularity conditions (support not depending on \( \theta \), differentiation under the integral allowed),
$$I(\theta) = E_\theta\left[\left(\frac{\partial}{\partial\theta}\log f(X; \theta)\right)^2\right] = -E_\theta\left[\frac{\partial^2}{\partial\theta^2}\log f(X; \theta)\right].$$A sample of \( n \) i.i.d. observations carries \( nI(\theta) \).
Under the same conditions, any unbiased estimator \( T \) of \( \tau(\theta) \) satisfies
$$\operatorname{Var}_\theta(T) \ge \frac{[\tau'(\theta)]^2}{nI(\theta)}.$$Equality holds iff the score is linear in \( T \): \( \frac{\partial}{\partial\theta}\log L = a(\theta)\left(T - \tau(\theta)\right) \), which happens only in a one-parameter exponential family with \( T \) its natural statistic (up to linear transformation).
| Model (one observation) | \( I(\theta) \) | CRLB for unbiased estimators of \( \theta \) |
|---|---|---|
| Bernoulli\( (p) \) | \( \frac{1}{p(1-p)} \) | \( \frac{p(1-p)}{n} \), attained by \( \bar{X} \) |
| Poisson\( (\lambda) \) | \( \frac{1}{\lambda} \) | \( \frac{\lambda}{n} \), attained by \( \bar{X} \) |
| \( N(\mu, \sigma^2) \), for \( \mu \) | \( \frac{1}{\sigma^2} \) | \( \frac{\sigma^2}{n} \), attained by \( \bar{X} \) |
| \( N(\mu, \sigma^2) \), for \( \sigma^2 \) | \( \frac{1}{2\sigma^4} \) | \( \frac{2\sigma^4}{n} \); \( S^2 \) has \( \frac{2\sigma^4}{n-1} \), not attained |
| Exponential, mean \( \theta \) | \( \frac{1}{\theta^2} \) | \( \frac{\theta^2}{n} \), attained by \( \bar{X} \) |
| Exponential, rate \( \lambda \) | \( \frac{1}{\lambda^2} \) | \( \frac{\lambda^2}{n} \); UMVUE \( \frac{n-1}{T} \) has variance \( \frac{\lambda^2}{n - 2} \), not attained |
A UMVUE need not attain the bound (the \( S^2 \) and \( \frac{n-1}{T} \) rows), so “UMVUE” and “efficient” are different properties: efficient implies UMVUE, not conversely.
Methods of Estimation
Method of moments
Equate the first \( k \) sample moments \( \frac{1}{n}\sum X_i^j \) to the population moments \( E(X^j) \) and solve for the \( k \) parameters. Simple, consistent (by the law of large numbers and continuity), often not efficient, and sometimes outside the parameter space.
Examples: \( U(0, \theta) \): \( E\,X = \frac{\theta}{2} \Rightarrow \hat{\theta} = 2\bar{X} \), unbiased with variance \( \frac{\theta^2}{3n} \). Gamma (shape \( \alpha \), rate \( \lambda \)): \( \hat{\alpha} = \frac{\bar{X}^2}{\hat{\sigma}^2} \), \( \hat{\lambda} = \frac{\bar{X}}{\hat{\sigma}^2} \) with \( \hat{\sigma}^2 = \frac{1}{n}\sum (X_i - \bar{X})^2 \).
Maximum likelihood
The likelihood \( L(\theta) = \prod f(x_i; \theta) \) is the joint density read as a function of \( \theta \). The MLE \( \hat{\theta} \) maximises it (usually via \( \ell = \log L \) and \( \ell'(\theta) = 0 \), checking it is a maximum).
| Model | MLE |
|---|---|
| Bernoulli, Poisson, exponential mean | \( \bar{X} \) |
| Exponential rate | \( 1/\bar{X} \) |
| \( N(\mu, \sigma^2) \) | \( \hat{\mu} = \bar{X} \), \( \hat{\sigma}^2 = \frac{1}{n}\sum (X_i - \bar{X})^2 \) (biased) |
| \( U(0, \theta) \) | \( X_{(n)} \) (likelihood \( \theta^{-n} \) is decreasing on \( \theta \ge x_{(n)} \)) |
| \( U(-\theta, \theta) \) | \( \max \lvert X_i \rvert \) |
| \( U(\theta - \frac{1}{2}, \theta + \frac{1}{2}) \) | any point of \( [X_{(n)} - \frac{1}{2},\, X_{(1)} + \frac{1}{2}] \): not unique |
| Shifted exponential \( e^{-(x - \theta)} \), \( x > \theta \) | \( X_{(1)} \) |
| Laplace location \( \frac{1}{2}e^{-\lvert x - \theta \rvert} \) | the sample median |
Properties of the MLE.
- Invariance: the MLE of \( g(\theta) \) is \( g(\hat{\theta}) \), for any function \( g \). (The MLE of \( \sigma \) is \( \hat{\sigma} \); of \( e^{-\lambda} \) for Poisson is \( e^{-\bar{X}} \).)
- It is a function of any sufficient statistic (when unique).
- Under regularity conditions it is consistent and asymptotically normal: \( \sqrt{n}(\hat{\theta} - \theta) \xrightarrow{d} N\left(0, \frac{1}{I(\theta)}\right) \), so asymptotically efficient.
- It may be biased, may not be unique, may not exist, and may not be found from \( \ell' = 0 \) (uniform models).
- If an efficient estimator exists, the likelihood equation gives it.
Least squares (minimise \( \sum (x_i - \mu_i(\theta))^2 \)) coincides with maximum likelihood when the errors are normal with constant variance.
Confidence Intervals
Pivotal method. Find a function \( Q(X, \theta) \) whose distribution is free of \( \theta \) (a pivot), choose \( a, b \) with \( P(a \le Q \le b) = 1 - \alpha \), and invert.
Notation: \( z_\alpha \), \( t_{\nu,\alpha} \), \( \chi^2_{\nu,\alpha} \) denote upper \( \alpha \) points.
| Parameter | Pivot | \( 100(1-\alpha)\% \) interval |
|---|---|---|
| \( \mu \), normal, \( \sigma \) known | \( \frac{\sqrt{n}(\bar{X} - \mu)}{\sigma} \sim N(0,1) \) | \( \bar{X} \pm z_{\alpha/2}\frac{\sigma}{\sqrt{n}} \) |
| \( \mu \), normal, \( \sigma \) unknown | \( \frac{\sqrt{n}(\bar{X} - \mu)}{S} \sim t_{n-1} \) | \( \bar{X} \pm t_{n-1,\alpha/2}\frac{S}{\sqrt{n}} \) |
| \( \sigma^2 \), normal | \( \frac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1} \) | \( \left[\frac{(n-1)S^2}{\chi^2_{n-1,\alpha/2}},\, \frac{(n-1)S^2}{\chi^2_{n-1,1-\alpha/2}}\right] \) |
| \( \mu_1 - \mu_2 \), equal unknown variances | pooled \( t_{n_1+n_2-2} \) | \( \bar{X}_1 - \bar{X}_2 \pm t\,S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}} \) |
| \( \sigma_1^2/\sigma_2^2 \) | \( \frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} \sim F_{n_1-1,n_2-1} \) | \( \left[\frac{S_1^2/S_2^2}{F_{\alpha/2}},\, \frac{S_1^2/S_2^2}{F_{1-\alpha/2}}\right] \) |
| \( p \), large \( n \) | \( \frac{\hat{p} - p}{\sqrt{p(1-p)/n}} \approx N(0,1) \) | \( \hat{p} \pm z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \) |
| Exponential mean \( \theta \) | \( \frac{2\sum X_i}{\theta} \sim \chi^2_{2n} \) | \( \left[\frac{2\sum X_i}{\chi^2_{2n,\alpha/2}},\, \frac{2\sum X_i}{\chi^2_{2n,1-\alpha/2}}\right] \) |
| \( U(0, \theta) \) | \( \frac{X_{(n)}}{\theta} \), CDF \( t^n \) on \( (0,1) \) | \( \left[X_{(n)},\, \frac{X_{(n)}}{\alpha^{1/n}}\right] \) (shortest) |
| any \( \theta \), large \( n \) | \( \sqrt{nI(\hat{\theta})}(\hat{\theta} - \theta) \approx N(0,1) \) | \( \hat{\theta} \pm \frac{z_{\alpha/2}}{\sqrt{nI(\hat{\theta})}} \) |
Common values: \( z_{0.05} = 1.645 \), \( z_{0.025} = 1.96 \), \( z_{0.005} = 2.576 \).
What moves the width. Width of the \( z \)-interval is \( 2z_{\alpha/2}\frac{\sigma}{\sqrt{n}} \): it grows with the confidence level and \( \sigma \), and shrinks like \( \frac{1}{\sqrt{n}} \). To estimate \( \mu \) within \( \pm E \): \( n \ge \left(\frac{z_{\alpha/2}\sigma}{E}\right)^2 \). For a proportion with no prior guess, use \( p = \frac{1}{2} \): \( n \ge \frac{z_{\alpha/2}^2}{4E^2} \).
Intervals and tests are duals. The \( 100(1 - \alpha)\% \) interval is the set of \( \theta_0 \) that a size-\( \alpha \) two-sided test would not reject. See Hypothesis Testing: MP, UMP and Likelihood Ratio Tests.
Memory Tricks
- “Sufficient squeezes, complete makes it unique.” Rao–Blackwell improves any unbiased estimator by conditioning on a sufficient statistic; completeness guarantees there is only one unbiased function of it.
- “Hunt the indicator.” When the support depends on \( \theta \), the MLE and sufficient statistic are order statistics, and Cramér–Rao does not apply.
- \( \frac{n+1}{n}X_{(n)} \). The uniform UMVUE: stretch the maximum by one expected gap.
- “Information adds.” \( n \) observations carry \( nI(\theta) \); the bound is \( \frac{\tau'^2}{nI} \).
- “MLE is invariant, unbiasedness is not.” \( g(\hat{\theta}) \) is always the MLE of \( g(\theta) \); \( g \) of an unbiased estimator is rarely unbiased.
Don’t Confuse
| This | Is not this | The difference |
|---|---|---|
| unbiased | consistent | neither implies the other |
| UMVUE | efficient (attains CRLB) | efficient \( \Rightarrow \) UMVUE; not conversely |
| sufficient | complete | \( (X_{(1)}, X_{(n)}) \) for \( U(\theta, \theta+1) \) is sufficient, not complete |
| MLE of \( \sigma^2 \): divisor \( n \) | UMVUE: divisor \( n - 1 \) | MLE is biased, UMVUE is not |
| 95% confidence interval | “95% probability \( \theta \) is in this interval” | the coverage is a property of the procedure |
| \( \chi^2_{\nu,\alpha} \) upper point | lower point | the variance interval puts the upper point in the lower limit |
Common Mistakes
Practice
All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.
Level 1: Foundation
- Write the MSE of an estimator in terms of its variance and bias.
- What is the MLE of \( \lambda \) for a Poisson sample?
- For i.i.d. Bernoulli\( (p) \), name a sufficient statistic.
Answers
- \( \text{MSE} = \operatorname{Var}(T) + [\text{bias}]^2 \).
- \( \bar{X} \).
- \( \sum X_i \) (complete sufficient).
Level 2: Part B Style
Practice Q1. For a random sample \( X_1, \dots, X_n \) from \( U(0, \theta) \), the maximum likelihood estimator of \( \theta \) is:
(a) \( 2\bar{X} \) (b) \( X_{(n)} \) (c) \( \frac{n+1}{n}X_{(n)} \) (d) \( X_{(1)} + X_{(n)} \)
Answer: (b)
\( L(\theta) = \theta^{-n}\mathbf{1}\{\theta \ge x_{(n)}\} \) is maximised at \( \theta = x_{(n)} \). Option (a) is the moment estimator; (c) is the UMVUE.
Practice Q2. The Cramér–Rao lower bound for the variance of unbiased estimators of \( \lambda \), based on a sample of size \( n \) from Poisson\( (\lambda) \), is:
(a) \( \frac{1}{n\lambda} \) (b) \( \frac{\lambda}{n} \) (c) \( \frac{\lambda^2}{n} \) (d) \( \frac{\sqrt{\lambda}}{n} \)
Answer: (b)
\( \log f = -\lambda + x\log\lambda - \log x! \); \( \frac{\partial^2}{\partial\lambda^2} = -\frac{x}{\lambda^2} \); \( I(\lambda) = \frac{1}{\lambda} \). Bound \( \frac{1}{nI} = \frac{\lambda}{n} \), attained by \( \bar{X} \).
Practice Q3. A population has \( \sigma = 10 \). The smallest sample size that estimates \( \mu \) to within \( \pm 2 \) with 95% confidence (using \( z_{0.025} = 1.96 \)) is:
(a) 25 (b) 49 (c) 96 (d) 97
Answer: (d)
\( n \ge \left(\frac{1.96 \times 10}{2}\right)^2 = 96.04 \), so \( n = 97 \). Rounding down to 96 would give a margin slightly above 2.
Practice Q4. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. Bernoulli\( (p) \) and \( T = \sum X_i \). The UMVUE of \( p(1 - p) \) is:
(a) \( \frac{T}{n}\left(1 - \frac{T}{n}\right) \) (b) \( \frac{T(n - T)}{n(n - 1)} \) (c) \( \frac{T(n - T)}{n^2} \) (d) \( \frac{T(T - 1)}{n(n - 1)} \)
Answer: (b)
\( E[T(n - T)] = n \cdot np - (npq + n^2p^2) = n(n-1)pq \). So \( \frac{T(n-T)}{n(n-1)} \) is unbiased and a function of the complete sufficient \( T \): UMVUE by Lehmann–Scheffé. Options (a) and (c) are the same biased plug-in; (d) is the UMVUE of \( p^2 \).
Level 3: Part C Style
Practice Q5. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. \( U(0, \theta) \), \( \theta > 0 \). Which statements are true?
(a) \( 2\bar{X} \) is unbiased for \( \theta \). (b) \( X_{(n)} \) is unbiased for \( \theta \). (c) \( \operatorname{Var}\left(\frac{n+1}{n}X_{(n)}\right) < \operatorname{Var}(2\bar{X}) \). (d) \( X_{(n)} \) is a consistent estimator of \( \theta \).
Answer: (a), (c) and (d)
- (a) True. \( E\,\bar{X} = \frac{\theta}{2} \).
- (b) False. \( E\,X_{(n)} = \frac{n\theta}{n+1} < \theta \).
- (c) True. \( \frac{\theta^2}{n(n+2)} < \frac{\theta^2}{3n} \iff n + 2 > 3 \iff n > 1 \).
- (d) True. \( P(X_{(n)} < \theta - \varepsilon) = \left(1 - \frac{\varepsilon}{\theta}\right)^n \to 0 \).
Practice Q6. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. \( N(\mu, \sigma^2) \), both parameters unknown. Which statements are true?
(a) \( (\bar{X}, S^2) \) is complete sufficient for \( (\mu, \sigma^2) \). (b) The MLE of \( \sigma^2 \) is unbiased. (c) \( S \) is an unbiased estimator of \( \sigma \). (d) \( \operatorname{Var}(S^2) \) is strictly greater than the Cramér–Rao bound \( \frac{2\sigma^4}{n} \).
Answer: (a) and (d)
- (a) True. Two-parameter exponential family with natural statistic \( (\sum X_i, \sum X_i^2) \), a one-to-one function of \( (\bar{X}, S^2) \).
- (b) False. \( E\left[\frac{1}{n}\sum (X_i - \bar{X})^2\right] = \frac{n-1}{n}\sigma^2 \).
- (c) False. \( E\,S < \sigma \) (Jensen).
- (d) True. \( \frac{2\sigma^4}{n - 1} > \frac{2\sigma^4}{n} \). \( S^2 \) is the UMVUE but not efficient.
Practice Q7. A random sample of 16 from a normal population gives \( \bar{x} = 50 \) and \( s = 8 \). Use \( t_{15,0.025} = 2.131 \) and \( z_{0.025} = 1.96 \). Which statements are true?
(a) The 95% confidence interval for \( \mu \) is about \( (45.74, 54.26) \). (b) If \( \sigma = 8 \) were known, the 95% interval would be \( (46.08, 53.92) \), narrower than in (a). (c) The 99% \( t \)-interval from the same data is narrower than the 95% one. (d) There is a 95% probability that \( \mu \) lies in the interval computed in (a).
Answer: (a) and (b)
- (a) True. \( 50 \pm 2.131 \times \frac{8}{4} = 50 \pm 4.262 \).
- (b) True. \( 50 \pm 1.96 \times 2 = 50 \pm 3.92 \).
- (c) False. Higher confidence needs a larger multiplier, so a wider interval.
- (d) False. \( \mu \) is fixed; the 95% describes how often the procedure covers \( \mu \) over repeated samples. (A Bayesian credible interval is the object that carries such a probability statement; see the Bayesian section.)
Practice Q8. \( X_1, \dots, X_n \) are i.i.d. exponential with rate \( \lambda \), \( n \ge 3 \), and \( T = \sum X_i \). Which statements are true?
(a) The MLE of \( \lambda \) is \( \frac{n}{T} \), and it is biased. (b) \( \frac{n - 1}{T} \) is the UMVUE of \( \lambda \). (c) The MLE of \( P(X_1 > 1) = e^{-\lambda} \) is \( e^{-n/T} \). (d) \( \bar{X} \) attains the Cramér–Rao bound for estimating \( \frac{1}{\lambda} \).
Answer: all four
- (a) True. \( \ell = n\log\lambda - \lambda T \), \( \hat{\lambda} = \frac{n}{T} \); \( E\left(\frac{1}{T}\right) = \frac{\lambda}{n-1} \), so \( E\,\hat{\lambda} = \frac{n\lambda}{n-1} \ne \lambda \).
- (b) True. Unbiased and a function of the complete sufficient \( T \).
- (c) True. Invariance of the MLE.
- (d) True. \( \operatorname{Var}\,\bar{X} = \frac{1}{n\lambda^2} \); the bound for \( \tau(\lambda) = \frac{1}{\lambda} \) is \( \frac{(1/\lambda^2)^2}{n/\lambda^2} = \frac{1}{n\lambda^2} \). The score is linear in \( \bar{X} \).
A Part C question can have every option correct; test each one rather than expecting a false one.
Quick Revision Table
| Idea | Fact |
|---|---|
| MSE | variance + bias\( ^2 \) |
| Consistency (sufficient) | bias \( \to 0 \) and variance \( \to 0 \) |
| Factorisation | \( f = g(T; \theta)h(x) \) |
| Exponential family | \( \sum t(X_i) \) complete sufficient |
| Rao–Blackwell | \( E(U \mid T) \) never worse |
| Lehmann–Scheffé | unbiased function of complete sufficient = unique UMVUE |
| Fisher information | \( -E[\partial^2_\theta \log f] \); \( n \) observations give \( nI \) |
| CRLB | \( \frac{\tau'(\theta)^2}{nI(\theta)} \); not for \( U(0, \theta) \) |
| MLE | invariant; asymptotically \( N(\theta, \frac{1}{nI}) \) |
| \( U(0, \theta) \) | MLE \( X_{(n)} \); UMVUE \( \frac{n+1}{n}X_{(n)} \); MoM \( 2\bar{X} \) |
| \( \mu \) interval | \( \bar{X} \pm z\sigma/\sqrt{n} \) or \( \bar{X} \pm tS/\sqrt{n} \) |
| \( \sigma^2 \) interval | \( \frac{(n-1)S^2}{\chi^2_{\text{upper}}} \) to \( \frac{(n-1)S^2}{\chi^2_{\text{lower}}} \) |
| Sample size | \( n \ge (z\sigma/E)^2 \) |
Key Takeaways
What’s Next
- Hypothesis Testing: MP, UMP and Likelihood Ratio Tests: the same sufficient statistics drive the best tests.
- Nonparametric Tests, Rank Correlation and Bayesian Inference: Bayes estimators and credible intervals as the other way to estimate.
- Sampling Distributions, Standard Errors and Order Statistics: the pivots used above.
- Back to the chapter overview.