Normal, Chi-Squared and Gamma Distributions

Here we show the classical example on how the normal, chi-squared, and gamma distributions are related.

If $$Z_1, \ldots, Z_n$$ are i.i.d. variables with the standard normal distribution, then $$\sum_{i=1}^n Z_i \sim \chi^2_n$$.

Let $$X_1, \ldots, X_n \sim \text{Normal}(\mu, \sigma^2)$$ be i.i.d. variables. Let $$\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$$, and $$S^2 = \frac{1}{n-1}\sum_{i=1}^n{(X_i – \overline{X})^2}$$. Then,

• $$\overline{X}$$ has normal distribution $$\text{Normal}(\mu, \sigma^2/n)$$;
• $$\overline{X}$$ and $$S^2$$ are independent;
• $$\frac{n-1}{\sigma^2}S^2 = \sum_{i=1}^n {(X_i-\overline{X})^2}/{\sigma^2} \sim \chi^2_{n-1}$$.

Note that, the chi-squared distribution has additivity. That is, for two independent variables $$X\sim \chi^2_k$$ and $$Y\sim \chi^2_m$$, we have $$X+Y \sim \chi^2_{k+m}$$.

The chi-squared distribution is a special case of the gamma distribution. A gamma distribution $$\text{Gamma}(k, \theta)$$ has two parameters: the shape parameter $$k$$ and the scale parameter $$\theta$$.
$\chi^2_k \equiv \text{Gamma}(\frac{k}{2}, 2)$

The gamma distribution has the following property. If $$X\sim \text{Gamma}(k, \theta)$$, then $$cX\sim \text{Gamma}(k, c\theta)$$ for any $$c > 0$$. Thus, we have
$S^2 \sim \text{Gamma}(\tfrac{n-1}{2}, \tfrac{2\sigma^2}{n-1}).$