Quadratic Forms of Random Variables
Quadratic Forms and Transformation
Let \(A = \{a_{ij}\}\) be an \(n\times n\) matrix. A quadratic function of \(n\) variables \(x = (x_1,\ldots, x_n)’\) is defined as
\[
f(x) = x’ A x = \sum_{i,j} a_{ij} x_i x_j.
\]
Without loss of generality, assume \(A\) is symmetric; otherwise replace \(A\) by \((A+A’)/2\).
Since \(A\) is symmetric, it has spectral decomposition
\[
A = Q’ \Lambda Q.
\]
\(\Lambda\) is diagonal and the diagonal elements \(\lambda_1, \ldots, \lambda_n\) are eigenvalues of \(A\). \(Q = (q_1, \ldots, q_n)\) is a orthogonal matrix with the eigenvectors \(q_i\) as columns.
Let \(y = Q’x = Q^{-1} x\). Then we have
\[
f (x) = x’A x = x’ Q \Lambda Q’ x = y’ \Lambda y = \sum_{i} \lambda_i y_i^2 =\sum_{i} ||q_i’ x||^2 .
\]
Random Variables
Let \(X= (X_1,\ldots, X_n)’\) be a random vector, with expectation \(\mu\) and covariance matrix \(\Sigma\):
\[
\mu = E[X] = (E[X_1], \ldots, E[X_n])
\]\[
\Sigma = E[(X-\mu) (X-\mu)’]
\]
The covariance matrix \(\Sigma\) is symmetric and positive semi-definite. This is because, for any vector \(b\) and \(Y= b’X\),
\[
0 \leq Var[Y] = Var[b’X] = b’ \Sigma b.
\]
Let \(A\) be a symmetric matrix, and define random variable \(Y = X’AX\). Then,
\[
E[Y] = E[X’A X] = tr(E[X’A X]) = E[ tr(X’A X) ] = E[ tr(A X X’) ]
\]\[
= tr(A E[ X X’ ]) = tr(A (\Sigma + \mu \mu’)) = tr(A\Sigma) + \mu’A\mu
\]