Skip to content

Linear Models and Multivariate Analysis Formula Sheet and Short Notes

One-page CSIR NET revision: Gauss–Markov and BLUE, F tests, ANOVA and EMS, regression diagnostics, multivariate normal, Wishart, correlation, PCA and LDA.

  • 7 min read
  • Updated
On this page · 9 sections
  1. The paper in one table
  2. Gauss–Markov model and linear hypotheses
  3. ANOVA and ANCOVA
  4. Regression
  5. Multivariate normal and quadratic forms
  6. Partial and multiple correlation
  7. PCA, discriminant, cluster, canonical
  8. Traps in one table
  9. Where to go next

Every formula, rule and trap from Gauss–Markov Models, BLUE and Linear Hypotheses, ANOVA, ANCOVA and Fixed, Random and Mixed Effects, Simple, Multiple and Logistic Regression, Multivariate Normal, Wishart and Quadratic Forms, Partial and Multiple Correlation and PCA, Discriminant, Cluster and Canonical Correlation Analysis, on one page.

The paper in one table

Per the NTA information bulletin for December 2026 (Mathematical Sciences, code 704):

PartGivenAttemptCorrectWrongMaximum
B4025+3−0.7575
C6020+4.75095

Part C: one or more options may be correct; marks only if you mark all correct options and no wrong one. No partial credit, no negative marking. Details on the Mathematical Sciences overview.

Gauss–Markov model and linear hypotheses

$$Y = X\beta + \varepsilon, \quad E\varepsilon = 0, \quad \operatorname{Var}\varepsilon = \sigma^2I, \quad \operatorname{rank}X = r.$$
ItemFact
Normal equations\( X^TX\beta = X^TY \), always consistent
Hat matrix\( P = X(X^TX)^-X^T \): symmetric, idempotent, \( \operatorname{tr} = r \), g-inverse free
Estimable \( \lambda^T\beta \)\( \lambda^T \in \) row space of \( X \); \( r \) independent ones
One-way effects\( \mu + \alpha_i \) and contrasts (\( \sum c_i = 0 \)) estimable; \( \mu \), \( \alpha_i \) alone not
BLUE\( \lambda^T\hat\beta \), \( \operatorname{Var} = \sigma^2\lambda^T(X^TX)^-\lambda \); needs no normality
GLS\( (X^TV^{-1}X)^{-1}X^TV^{-1}Y \) if \( \operatorname{Var} = \sigma^2V \)
\( \sigma^2 \)\( s^2 = \frac{Y^T(I - P)Y}{n - r} \) unbiased; MLE divides by \( n \)
Normal theory\( \text{SSE}/\sigma^2 \sim \chi^2_{n - r} \), independent of \( \hat Y \)
F test, \( H_0: L\beta = c \)\( \frac{(\text{SSE}_H - \text{SSE})/q}{\text{SSE}/(n - r)} \sim F_{q, n - r} \)

ANOVA and ANCOVA

Designdf
one-way, \( k \) groups, \( N \) obstreatments \( k - 1 \), error \( N - k \)
two-way, one per cellA \( a - 1 \), B \( b - 1 \), error \( (a - 1)(b - 1) \)
two-way, \( n \) per cellA \( a - 1 \), B \( b - 1 \), AB \( (a - 1)(b - 1) \), error \( ab(n - 1) \)
ANCOVA one-wayerror \( N - k - 1 \)
ModelF denominators
all fixedMSE for A, B, AB
all randomA and B over MSAB; AB over MSE
A fixed, B random (restricted)A over MSAB; B over MSE; AB over MSE
A fixed, B random (unrestricted)A over MSAB; B over MSAB; AB over MSE
  • Random one-way: \( E(\text{MSB}) = \sigma^2 + n\sigma_a^2 \); \( \hat\sigma_a^2 = \frac{\text{MSB} - \text{MSE}}{n} \) (can be negative); intraclass \( \rho_I = \frac{\sigma_a^2}{\sigma_a^2 + \sigma^2} \).
  • ANCOVA: \( \hat\beta = \frac{E_{xy}}{E_{xx}} \); adjusted error SS \( E_{yy} - \frac{E_{xy}^2}{E_{xx}} \); adjusted mean \( \bar y_i - \hat\beta(\bar x_i - \bar x) \); assumes equal slopes and a covariate unaffected by treatment.

Regression

ItemFact
Simple\( \hat\beta_1 = \frac{S_{xy}}{S_{xx}} \), \( \hat\beta_0 = \bar y - \hat\beta_1\bar x \), \( \operatorname{Var}\hat\beta_1 = \frac{\sigma^2}{S_{xx}} \), df \( n - 2 \)
\( R^2 \) (simple)\( r^2 \); slope \( t^2 = F \)
Intervals at \( x_0 \)mean: \( s\sqrt{\frac{1}{n} + \frac{(x_0 - \bar x)^2}{S_{xx}}} \); new obs: add 1 under the root
Multiple\( \hat\beta = (X^TX)^{-1}X^TY \), df \( n - p \) (\( p \) includes intercept)
Adjusted \( R^2 \)\( 1 - (1 - R^2)\frac{n - 1}{n - p} \)
Overall F\( \frac{R^2/(p - 1)}{(1 - R^2)/(n - p)} \)
Leverage\( \sum h_{ii} = p \), \( \frac{1}{n} \le h_{ii} \le 1 \), flag \( > \frac{2p}{n} \); simple: \( \frac{1}{n} + \frac{(x_i - \bar x)^2}{S_{xx}} \)
Residuals\( \operatorname{Var}e_i = \sigma^2(1 - h_{ii}) \); correlated; sum to 0 with intercept
Cook’s distance\( \frac{r_i^2}{p}\cdot\frac{h_{ii}}{1 - h_{ii}} \)
VIF\( \frac{1}{1 - R_j^2} \)
Durbin–Watson\( d \approx 2(1 - \hat\rho) \in [0, 4] \)
Logistic\( \ln\frac{p}{1 - p} = x^T\beta \); \( e^{\beta_j} \) = odds ratio; MLE by IRLS; score \( X^T(y - \hat p) = 0 \)

Multivariate normal and quadratic forms

ItemFact
Definitionevery \( a^TX \) normal
Linear map\( AX + b \sim N(A\mu + b, A\Sigma A^T) \)
Independencejointly normal: \( \Sigma_{12} = 0 \iff \) independent
Conditional\( N(\mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(x_2 - \mu_2), \; \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}) \)
Bivariatemean \( \mu_1 + \rho\frac{\sigma_1}{\sigma_2}(x_2 - \mu_2) \), variance \( \sigma_1^2(1 - \rho^2) \)
Conditional independence\( (\Sigma^{-1})_{ij} = 0 \)
Mahalanobis\( (X - \mu)^T\Sigma^{-1}(X - \mu) \sim \chi^2_p \)
Mean of a form\( E(X^TAX) = \operatorname{tr}(A\Sigma) + \mu^TA\mu \) (any distribution)
Chi-square\( A\Sigma \) idempotent; df \( = \operatorname{rank}A \); non-centrality \( \mu^TA\mu \) (or half, by convention)
Independence of forms\( A\Sigma B = 0 \); linear \( LX \) and \( X^TAX \): \( L\Sigma A = 0 \)
Cochran\( \sum A_i = I \), ranks add to \( n \) \( \iff \) independent \( \chi^2_{r_i} \)
Wishart\( \sum Z_iZ_i^T \sim W_p(n, \Sigma) \); \( E = n\Sigma \); \( (n - 1)S \sim W_p(n - 1, \Sigma) \)
Wishart extras\( \frac{a^TWa}{a^T\Sigma a} \sim \chi^2_n \); invertible iff \( n \ge p \); \( E(W^{-1}) = \frac{\Sigma^{-1}}{n - p - 1} \)
Hotelling\( \frac{n - p}{p(n - 1)}\, n(\bar X - \mu_0)^TS^{-1}(\bar X - \mu_0) \sim F_{p, n - p} \)

Partial and multiple correlation

$$r_{12\cdot3} = \frac{r_{12} - r_{13}r_{23}}{\sqrt{(1 - r_{13}^2)(1 - r_{23}^2)}}, \qquad R^2_{1\cdot23} = \frac{r_{12}^2 + r_{13}^2 - 2r_{12}r_{13}r_{23}}{1 - r_{23}^2}.$$
  • \( 1 - R^2_{1\cdot23} = (1 - r_{12}^2)(1 - r_{13\cdot2}^2) \).
  • From \( P = R^{-1} \): \( r_{ij\cdot\text{rest}} = -\frac{p^{ij}}{\sqrt{p^{ii}p^{jj}}} \), \( R^2_{i\cdot\text{rest}} = 1 - \frac{1}{p^{ii}} \).
  • \( 0 \le R \le 1 \); \( R_{1\cdot23} \ge \lvert r_{12} \rvert, \lvert r_{13} \rvert \); \( R = 0 \iff r_{12} = r_{13} = 0 \).
  • Feasible iff \( 1 - r_{12}^2 - r_{13}^2 - r_{23}^2 + 2r_{12}r_{13}r_{23} \ge 0 \).
  • Equicorrelation \( \rho \): \( r_{12\cdot3} = \frac{\rho}{1 + \rho} \), \( R^2_{1\cdot23} = \frac{2\rho^2}{1 + \rho} \).
  • Tests: partial of order \( k \): \( t = \frac{r\sqrt{n - 2 - k}}{\sqrt{1 - r^2}} \sim t_{n - 2 - k} \); multiple with \( k \) predictors: \( F_{k, n - k - 1} \).

PCA, discriminant, cluster, canonical

TechniqueKey fact
PCA\( Y_j = e_j^T(X - \mu) \), \( \operatorname{Var} = \lambda_j \); share \( \frac{\lambda_j}{\operatorname{tr}\Sigma} \); depends on units
\( 2 \times 2 \) correlationeigenvalues \( 1 \pm \rho \), PC1 \( \propto Z_1 + Z_2 \)
Equicorrelation\( 1 + (p - 1)\rho \) and \( 1 - \rho \) (\( p - 1 \) times)
Fisher LDF\( a = \Sigma^{-1}(\mu_1 - \mu_2) \); to \( \pi_1 \) if \( a^Tx \ge \frac{1}{2}a^T(\mu_1 + \mu_2) \)
Errors\( \Phi(-\Delta/2) \), \( \Delta^2 = (\mu_1 - \mu_2)^T\Sigma^{-1}(\mu_1 - \mu_2) \)
Priors, costscut-off shifts by \( \ln\frac{c(1 \mid 2)p_2}{c(2 \mid 1)p_1} \)
Several groupseigenvectors of \( W^{-1}B \), at most \( \min(g - 1, p) \)
Linkagessingle min (chaining), complete max (compact), average mean, Ward WSS
k-meansminimises WSS, local optimum, \( k \) fixed in advance
Canonical\( \rho_k^2 \) = eigenvalues of \( \Sigma_{11}^{-1}\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} \); \( \min(p, q) \) pairs; invariant within sets
\( q = 1 \)canonical correlation = multiple correlation

Traps in one table

StatementVerdict
Gauss–Markov needs normal errorsfalse
Every \( \beta_j \) is estimable when \( r < p \)false
\( s^2 = \text{SSE}/(n - p) \) in a rank-deficient modelfalse: \( n - r \)
Residuals are independent with variance \( \sigma^2 \)false
Adding a predictor can lower \( R^2 \)false (adjusted \( R^2 \) can fall)
\( e^{\beta_j} \) in logistic regression is a ratio of probabilitiesfalse: odds ratio
Normal and uncorrelated implies independentfalse without joint normality
\( \Sigma_{13} = 0 \) implies \( X_1 \perp X_3 \mid X_2 \)false: look at \( \Sigma^{-1} \)
\( \hat\sigma_a^2 \) is always non-negativefalse
Fixed factor crossed with a random one is tested against MSEfalse: MSAB
PCA is unchanged by rescaling variablesfalse (true for canonical correlations)
\( R_{1\cdot23} \) can be smaller than \( \lvert r_{12} \rvert \)false

Where to go next

Advertisement
Advertisement