Normal, Chi-Squared and Gamma Distributions
Here we show the classical example on how the normal, chi-squared, and gamma distributions are related.
If \(Z_1, \ldots, Z_n\) are i.i.d. variables with the standard normal distribution, then \(\sum_{i=1}^n Z_i \sim \chi^2_n\).
Let \(X_1, \ldots, X_n \sim \text{Normal}(\mu, \sigma^2)\) be i.i.d. variables. Let \(\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i\), and \(S^2 = \frac{1}{n-1}\sum_{i=1}^n{(X_i – \overline{X})^2}\). Then,
- \(\overline{X}\) has normal distribution \(\text{Normal}(\mu, \sigma^2/n)\);
- \(\overline{X}\) and \(S^2\) are independent;
- \(\frac{n-1}{\sigma^2}S^2 = \sum_{i=1}^n {(X_i-\overline{X})^2}/{\sigma^2} \sim \chi^2_{n-1}\).
Note that, the chi-squared distribution has additivity. That is, for two independent variables \(X\sim \chi^2_k\) and \(Y\sim \chi^2_m\), we have \(X+Y \sim \chi^2_{k+m}\).
The chi-squared distribution is a special case of the gamma distribution. A gamma distribution \(\text{Gamma}(k, \theta)\) has two parameters: the shape parameter \(k\) and the scale parameter \(\theta\).
\[
\chi^2_k \equiv \text{Gamma}(\frac{k}{2}, 2)
\]
The gamma distribution has the following property. If \(X\sim \text{Gamma}(k, \theta)\), then \(cX\sim \text{Gamma}(k, c\theta)\) for any \(c > 0\). Thus, we have
\[
S^2 \sim \text{Gamma}(\tfrac{n-1}{2}, \tfrac{2\sigma^2}{n-1}).
\]