# Quadratic Forms of Random Variables

Let $$A = \{a_{ij}\}$$ be an $$n\times n$$ matrix. A quadratic function of $$n$$ variables $$x = (x_1,\ldots, x_n)’$$ is defined as
$f(x) = x' A x = \sum_{i,j} a_{ij} x_i x_j.$
Without loss of generality, assume $$A$$ is symmetric; otherwise replace $$A$$ by $$(A+A’)/2$$.

Since $$A$$ is symmetric, it has spectral decomposition
$A = Q' \Lambda Q.$
$$\Lambda$$ is diagonal and the diagonal elements $$\lambda_1, \ldots, \lambda_n$$ are eigenvalues of $$A$$. $$Q = (q_1, \ldots, q_n)$$ is a orthogonal matrix with the eigenvectors $$q_i$$ as columns.

Let $$y = Q’x = Q^{-1} x$$. Then we have
$f (x) = x'A x = x' Q \Lambda Q' x = y' \Lambda y = \sum_{i} \lambda_i y_i^2 =\sum_{i} ||q_i' x||^2 .$

### Random Variables

Let $$X= (X_1,\ldots, X_n)’$$ be a random vector, with expectation $$\mu$$ and covariance matrix $$\Sigma$$:
$\mu = E[X] = (E[X_1], \ldots, E[X_n])$$\Sigma = E[(X-\mu) (X-\mu)']$

The covariance matrix $$\Sigma$$ is symmetric and positive semi-definite. This is because, for any vector $$b$$ and $$Y= b’X$$,
$0 \leq Var[Y] = Var[b'X] = b’ \Sigma b.$

Let $$A$$ be a symmetric matrix, and define random variable $$Y = X’AX$$. Then,
$E[Y] = E[X'A X] = tr(E[X'A X]) = E[ tr(X'A X) ] = E[ tr(A X X') ]$$= tr(A E[ X X' ]) = tr(A (\Sigma + \mu \mu’)) = tr(A\Sigma) + \mu’A\mu$