Skip to content

Partial and Multiple Correlation

CSIR NET statistics: partial and multiple correlation coefficients, three-variable formulas, the inverse correlation matrix, bounds, and t and F tests.

  • 16 min read
  • Updated
On this page · 15 sections
  1. Prerequisites
  2. The Hook: Ice Cream and Drowning
  3. Notation
  4. Partial Correlation
  5. Multiple Correlation
  6. Everything From the Inverse Correlation Matrix
  7. Which Correlations Are Possible?
  8. Tests
  9. Memory Tricks
  10. Don’t Confuse
  11. Common Mistakes
  12. Practice
  13. Quick Revision Table
  14. Key Takeaways
  15. What’s Next

Prerequisites

The Hook: Ice Cream and Drowning

Notation

For variables \( X_1, \dots, X_p \) with correlations \( r_{ij} \) (sample) or \( \rho_{ij} \) (population):

  • \( r_{12\cdot3} \): partial correlation between \( X_1 \) and \( X_2 \), eliminating \( X_3 \). The variables after the dot are held fixed (removed). The number of variables after the dot is the order: \( r_{12} \) is zero order, \( r_{12\cdot3} \) first order, \( r_{12\cdot34} \) second order.
  • \( R_{1\cdot23} \): multiple correlation of \( X_1 \) on \( X_2 \) and \( X_3 \). The variable before the dot is predicted, those after are predictors.

Partial Correlation

Partial correlation

Let \( e_{1\cdot3} \) be the residual of \( X_1 \) after its least squares linear regression on \( X_3 \), and \( e_{2\cdot3} \) the residual of \( X_2 \) on \( X_3 \). Then

$$r_{12\cdot3} = \operatorname{corr}(e_{1\cdot3}, e_{2\cdot3}).$$

More generally, \( r_{ij\cdot S} \) is the correlation of the residuals of \( X_i \) and \( X_j \) after regression on the set \( S \).

Working out the residual covariance gives the three-variable formula:

$$\boxed{r_{12\cdot3} = \frac{r_{12} - r_{13}r_{23}}{\sqrt{(1 - r_{13}^2)(1 - r_{23}^2)}}}$$

and the recursion for higher orders, peeling off one variable at a time:

$$r_{12\cdot34} = \frac{r_{12\cdot3} - r_{14\cdot3}r_{24\cdot3}}{\sqrt{(1 - r_{14\cdot3}^2)(1 - r_{24\cdot3}^2)}}.$$

Reading the formula.

  • \( r_{12\cdot3} = 0 \) iff \( r_{12} = r_{13}r_{23} \): the whole association of \( X_1 \) and \( X_2 \) runs through \( X_3 \).
  • \( r_{12\cdot3} \) can have the opposite sign to \( r_{12} \), and can be larger in magnitude.
  • If \( r_{13} = 0 \) or \( r_{23} = 0 \), the numerator is \( r_{12} \) but the denominator is still \( < 1 \) in general, so \( \lvert r_{12\cdot3} \rvert \ge \lvert r_{12} \rvert \).
  • \( -1 \le r_{12\cdot3} \le 1 \), like any correlation.

Population meaning for normal data

For a jointly normal vector, the population partial correlation \( \rho_{12\cdot3} \) is the correlation in the conditional distribution of \( (X_1, X_2) \) given \( X_3 = x_3 \), and it does not depend on \( x_3 \). So for normal data

$$\rho_{12\cdot3} = 0 \iff X_1 \perp X_2 \mid X_3.$$

For non-normal data a zero partial correlation only says the residuals of the linear regressions are uncorrelated.

Multiple Correlation

Multiple correlation
\( R_{1\cdot23\ldots p} \) is the correlation between \( X_1 \) and its least squares linear predictor \( \hat X_1 \) from \( X_2, \dots, X_p \). Equivalently, it is the maximum of \( \operatorname{corr}(X_1, a_2X_2 + \dots + a_pX_p) \) over all coefficients. It always lies in \( [0, 1] \), and \( R^2_{1\cdot23\ldots p} \) is the coefficient of determination of that regression.

Three-variable formula:

$$\boxed{R^2_{1\cdot23} = \frac{r_{12}^2 + r_{13}^2 - 2r_{12}r_{13}r_{23}}{1 - r_{23}^2}}$$

and the product identity linking it to partial correlations:

$$1 - R^2_{1\cdot23} = (1 - r_{12}^2)(1 - r_{13\cdot2}^2), \qquad 1 - R^2_{1\cdot23\ldots p} = (1 - r_{12}^2)(1 - r_{13\cdot2}^2)(1 - r_{14\cdot23}^2)\cdots.$$

Each factor is the proportion of variance left unexplained after adding one more predictor. The residual (partial) variance is \( \sigma^2_{1\cdot23\ldots p} = \sigma_1^2(1 - R^2_{1\cdot23\ldots p}) \).

Properties.

PropertyStatement
range\( 0 \le R \le 1 \) (never negative)
dominates simple correlations\( R_{1\cdot23} \ge \max(\lvert r_{12} \rvert, \lvert r_{13} \rvert) \)
monotoneadding predictors never decreases \( R \)
zero\( R_{1\cdot23} = 0 \iff r_{12} = r_{13} = 0 \)
one\( R_{1\cdot23} = 1 \iff X_1 \) is exactly a linear function of \( X_2, X_3 \)
uncorrelated predictorsif \( r_{23} = 0 \), \( R^2_{1\cdot23} = r_{12}^2 + r_{13}^2 \)
asymmetric\( R_{1\cdot23} \ne R_{2\cdot13} \) in general

Everything From the Inverse Correlation Matrix

Let \( R = (r_{ij}) \) be the \( p \times p \) correlation matrix (positive definite) and \( P = R^{-1} = (p^{ij}) \). Then

$$\boxed{r_{ij\cdot\text{rest}} = -\frac{p^{ij}}{\sqrt{p^{ii}p^{jj}}}, \qquad R^2_{i\cdot\text{rest}} = 1 - \frac{1}{p^{ii}} = 1 - \frac{\lvert R \rvert}{\lvert R_{ii} \rvert},}$$

where “rest” means all the other variables and \( \lvert R_{ii} \rvert \) is the minor obtained by deleting row and column \( i \). With the covariance matrix instead, \( \Sigma^{-1} = (\sigma^{ij}) \), the partial correlation formula is unchanged (the scales cancel), while \( R^2_{i\cdot\text{rest}} = 1 - \frac{1}{\sigma_{ii}\sigma^{ii}} \). For the diagonal: \( \frac{1}{p^{ii}} = 1 - R^2_{i\cdot\text{rest}} \), and \( p^{ii} \) is the variance inflation factor of \( X_i \) when all the other variables are used as predictors together with it.

Which Correlations Are Possible?

A correlation matrix must be positive semidefinite. For three variables this is

$$\lvert R \rvert = 1 - r_{12}^2 - r_{13}^2 - r_{23}^2 + 2r_{12}r_{13}r_{23} \ge 0,$$

which, for given \( r_{12} \) and \( r_{13} \), confines \( r_{23} \) to

$$r_{12}r_{13} - \sqrt{(1 - r_{12}^2)(1 - r_{13}^2)} \;\le\; r_{23} \;\le\; r_{12}r_{13} + \sqrt{(1 - r_{12}^2)(1 - r_{13}^2)}.$$

Example: \( r_{12} = r_{13} = 0.9 \) forces \( 0.62 \le r_{23} \le 1 \). Correlation is not transitive in general (with \( r_{12} = r_{13} = 0.5 \), \( r_{23} \) can be anything from \( -0.5 \) to 1), but strong correlations force a positive one.

Equicorrelation. If every \( r_{ij} = \rho \) among three variables: \( r_{12\cdot3} = \frac{\rho}{1 + \rho} \), \( R^2_{1\cdot23} = \frac{2\rho^2}{1 + \rho} \), and \( \lvert R \rvert = (1 - \rho)^2(1 + 2\rho) \), which needs \( \rho \ge -\frac{1}{2} \).

Tests

For a sample of size \( n \) from a multivariate normal population:

HypothesisStatisticNull distribution
\( \rho_{12} = 0 \)\( t = \frac{r\sqrt{n - 2}}{\sqrt{1 - r^2}} \)\( t_{n - 2} \)
\( \rho_{12\cdot S} = 0 \), \( k \) variables in \( S \)\( t = \frac{r_{12\cdot S}\sqrt{n - 2 - k}}{\sqrt{1 - r_{12\cdot S}^2}} \)\( t_{n - 2 - k} \)
\( R_{1\cdot2\ldots(k+1)} = 0 \) (\( k \) predictors)\( F = \frac{R^2/k}{(1 - R^2)/(n - k - 1)} \)\( F_{k, n - k - 1} \)

A partial correlation of order \( k \) behaves like a simple correlation computed from \( n - k \) observations: each variable partialled out costs one degree of freedom. The multiple correlation test is the overall regression F test.

graph TD
    A["Given r12, r13, r23"] --> B{"What is asked?"}
    B -->|"partial r12.3"| C["(r12 - r13 r23) / sqrt((1 - r13^2)(1 - r23^2))"]
    B -->|"multiple R1.23"| D["R^2 = (r12^2 + r13^2 - 2 r12 r13 r23) / (1 - r23^2)"]
    B -->|"many variables"| E["Invert R: partial = -p_ij / sqrt(p_ii p_jj),
R^2 = 1 - 1/p_ii"] B -->|"is it possible?"| F["Check det R >= 0"]

Memory Tricks

  • Partial: “subtract the path through 3, then rescale.” Numerator \( r_{12} - r_{13}r_{23} \); denominator uses the two correlations with 3.
  • Multiple: “sum of squares minus twice the triple product, over one minus the predictors’ r².”
  • Precision matrix: minus sign for partials, reciprocal for R². \( -\frac{p^{ij}}{\sqrt{p^{ii}p^{jj}}} \) and \( 1 - \frac{1}{p^{ii}} \).
  • Each partialled variable costs one df. \( t_{n - 2 - k} \).
  • R never loses. \( R_{1\cdot23} \ge \lvert r_{12} \rvert \), \( \lvert r_{13} \rvert \), and it is never negative.

Don’t Confuse

ThisIs not thisThe difference
\( r_{12\cdot3} \) (partial)\( R_{1\cdot23} \) (multiple)partial: two variables, a third removed; multiple: one variable against a set
\( R_{1\cdot23} \)\( R_{2\cdot13} \)different dependent variable, different value
partial correlation \( = 0 \) (normal data)partial correlation \( = 0 \) (any data)conditional independence only under normality
\( r_{12\cdot3} \) with sign of \( r_{12} \)\( r_{12\cdot3} \)it can reverse sign
multiple correlation \( R \)simple correlation \( r \)\( 0 \le R \le 1 \) always, while \( r \in [-1, 1] \); \( R \) is the correlation with the best linear predictor, whose sign is chosen to make it non-negative
\( \Sigma^{-1} \) entries\( \Sigma \) entriesthe inverse gives partials and \( R^2 \); \( \Sigma \) gives simple covariances
partial correlationsemi-partial (part) correlationthe part correlation removes \( X_3 \) from only one of the two variables

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.

Level 1: Foundation

  1. Write \( r_{12\cdot3} \) in terms of the simple correlations.
  2. What does \( R_{1\cdot23} = 0 \) say about \( r_{12} \) and \( r_{13} \)?
  3. If \( r_{12} = r_{13}r_{23} \), what is \( r_{12\cdot3} \)?
Answers
  1. \( \frac{r_{12} - r_{13}r_{23}}{\sqrt{(1 - r_{13}^2)(1 - r_{23}^2)}} \).
  2. Both are zero.
  3. Zero (the numerator vanishes).

Level 2: Part B Style

Practice Q1. Given \( r_{12} = 0.6 \), \( r_{13} = 0.5 \), \( r_{23} = 0.4 \), the partial correlation \( r_{12\cdot3} \) is closest to:

(a) 0.40 (b) 0.50 (c) 0.56 (d) 0.66

Answer: (b)

\( \frac{0.6 - 0.2}{\sqrt{0.75 \times 0.84}} = \frac{0.4}{0.794} \approx 0.504 \). Option (a) forgets the denominator; (d) is \( R_{1\cdot23} \).

Practice Q2. With the same correlations, \( R^2_{1\cdot23} \) is closest to:

(a) 0.36 (b) 0.44 (c) 0.61 (d) 0.66

Answer: (b)

\( \frac{0.36 + 0.25 - 2(0.6)(0.5)(0.4)}{1 - 0.16} = \frac{0.37}{0.84} \approx 0.440 \). Option (c) is \( r_{12}^2 + r_{13}^2 \), which ignores the correlation between the predictors.

Practice Q3. Given \( r_{12} = 0.3 \), \( r_{13} = 0.8 \), \( r_{23} = 0.6 \), the partial correlation \( r_{12\cdot3} \) is:

(a) 0.375 (b) \( -0.375 \) (c) \( -0.18 \) (d) 0.3

Answer: (b)

\( \frac{0.3 - 0.48}{\sqrt{(1 - 0.64)(1 - 0.36)}} = \frac{-0.18}{0.6 \times 0.8} = -0.375 \). The sign reverses once \( X_3 \) is held fixed.

Practice Q4. From a normal sample of \( n = 30 \), the first-order partial correlation \( r_{12\cdot3} = 0.5 \). The t statistic for \( H_0: \rho_{12\cdot3} = 0 \) and its degrees of freedom are:

(a) 3.0 on 27 df (b) 3.06 on 28 df (c) 2.6 on 27 df (d) 3.0 on 28 df

Answer: (a)

\( t = \frac{0.5\sqrt{30 - 2 - 1}}{\sqrt{1 - 0.25}} = \frac{0.5\sqrt{27}}{\sqrt{0.75}} = 0.5\sqrt{36} = 3.0 \), on \( 27 \) df. Option (b) treats it as a simple correlation, with \( n - 2 = 28 \) df.

Level 3: Part C Style

Practice Q5. Three variables have all pairwise correlations equal to \( 0.5 \). Which statements are true?

(a) \( r_{12\cdot3} = \frac{1}{3} \). (b) \( R^2_{1\cdot23} = \frac{1}{3} \). (c) \( R_{1\cdot23} < r_{12} \). (d) The correlation matrix is singular.

Answer: (a) and (b)
  • (a) True. \( \frac{0.5 - 0.25}{1 - 0.25} = \frac{1}{3} \) (generally \( \frac{\rho}{1 + \rho} \)).
  • (b) True. \( \frac{0.25 + 0.25 - 0.25}{0.75} = \frac{1}{3} \) (generally \( \frac{2\rho^2}{1 + \rho} \)).
  • (c) False. \( R_{1\cdot23} = \frac{1}{\sqrt{3}} \approx 0.577 > 0.5 \); a multiple correlation is never below a simple one with the same dependent variable.
  • (d) False. \( \lvert R \rvert = (1 - 0.5)^2(1 + 1) = 0.5 \ne 0 \).

Practice Q6. The inverse of the correlation matrix of \( (X_1, X_2, X_3) \) is \( P = \frac{1}{47}\begin{pmatrix} 84 & -40 & -26 \\ -40 & 75 & -10 \\ -26 & -10 & 64 \end{pmatrix} \). Which statements are true?

(a) \( R^2_{1\cdot23} = \frac{37}{84} \). (b) \( r_{12\cdot3} = \frac{40}{\sqrt{84 \times 75}} \approx 0.504 \). (c) \( r_{12\cdot3} \) is negative because \( p^{12} \) is negative. (d) The variance inflation factor of \( X_1 \) in the regression of \( X_3 \) on \( X_1, X_2 \) is \( \frac{84}{47} \).

Answer: (a) and (b)

(This \( P \) is the inverse of the matrix with \( r_{12} = 0.6 \), \( r_{13} = 0.5 \), \( r_{23} = 0.4 \).)

  • (a) True. \( 1 - \frac{1}{p^{11}} = 1 - \frac{47}{84} = \frac{37}{84} \approx 0.440 \).
  • (b) True. \( -\frac{p^{12}}{\sqrt{p^{11}p^{22}}} = \frac{40/47}{\sqrt{84 \times 75}/47} \).
  • (c) False. The formula carries a minus sign: a negative off-diagonal entry of the precision matrix means a positive partial correlation.
  • (d) False. \( p^{11} = \frac{84}{47} \) is the VIF of \( X_1 \) when all the other variables, \( X_2 \) and \( X_3 \), are its fellow predictors. In the regression of \( X_3 \) on \( X_1, X_2 \), the only other predictor is \( X_2 \), so the VIF of \( X_1 \) is \( \frac{1}{1 - r_{12}^2} = \frac{1}{0.64} = 1.5625 \).

Practice Q7. Suppose \( r_{12} = r_{13} = 0.9 \). Which statements are true?

(a) \( r_{23} = 0 \) is possible. (b) \( r_{23} \) must be at least 0.62. (c) If \( r_{23} = 0.81 \), then \( r_{23\cdot1} = 0 \). (d) If \( r_{23} = 0.81 \), then \( R^2_{1\cdot23} > 0.81 \).

Answer: (b), (c) and (d)
  • (a) False. \( \lvert R \rvert = 1 - 0.81 - 0.81 = -0.62 < 0 \).
  • (b) True. \( r_{23} \ge 0.81 - \sqrt{0.19 \times 0.19} = 0.62 \).
  • (c) True. \( r_{23\cdot1} = \frac{0.81 - 0.9 \times 0.9}{0.19} = 0 \): \( X_2 \) and \( X_3 \) are related only through \( X_1 \).
  • (d) True. \( R^2_{1\cdot23} = \frac{0.81 + 0.81 - 2(0.81)(0.81)}{1 - 0.6561} = \frac{0.3078}{0.3439} \approx 0.895 \). It must be at least \( r_{12}^2 = 0.81 \), and here it is strictly larger.

Quick Revision Table

IdeaFact
Partial (definition)correlation of residuals after linear regression on the removed variables
\( r_{12\cdot3} \)\( \frac{r_{12} - r_{13}r_{23}}{\sqrt{(1 - r_{13}^2)(1 - r_{23}^2)}} \)
Recursion\( r_{12\cdot34} \) from first-order partials with 4 removed
\( R^2_{1\cdot23} \)\( \frac{r_{12}^2 + r_{13}^2 - 2r_{12}r_{13}r_{23}}{1 - r_{23}^2} \)
Product identity\( 1 - R^2_{1\cdot23} = (1 - r_{12}^2)(1 - r_{13\cdot2}^2) \)
From \( P = R^{-1} \)\( r_{ij\cdot\text{rest}} = -\frac{p^{ij}}{\sqrt{p^{ii}p^{jj}}} \), \( R^2_{i\cdot\text{rest}} = 1 - \frac{1}{p^{ii}} \)
Bounds\( 0 \le R \le 1 \); \( R_{1\cdot23} \ge \lvert r_{12} \rvert, \lvert r_{13} \rvert \)
Feasibility\( 1 - r_{12}^2 - r_{13}^2 - r_{23}^2 + 2r_{12}r_{13}r_{23} \ge 0 \)
Equicorrelation\( r_{12\cdot3} = \frac{\rho}{1 + \rho} \), \( R^2_{1\cdot23} = \frac{2\rho^2}{1 + \rho} \)
Testspartial of order \( k \): \( t_{n - 2 - k} \); multiple: \( F_{k, n - k - 1} \)
Normal data\( \rho_{12\cdot3} = 0 \iff X_1 \perp X_2 \mid X_3 \)

Key Takeaways

What’s Next