Quadratic form (statistics)

In multivariate statistics, if \varepsilon is a vector of n random variables, and \Lambda is an n -dimensional symmetric matrix, then the scalar quantity \varepsilon^T\Lambda\varepsilon is known as a quadratic form in \varepsilon .

Uses your system voice; voice and rate in settings.

≈ 4 min read

Cite, export and learn8 citation formats · 8 exports · lesson and quiz

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In multivariate statistics, if \varepsilon is a vector of n random variables, and \Lambda is an n -dimensional symmetric matrix, then the scalar quantity \varepsilon^T\Lambda\varepsilon is known as a quadratic form in \varepsilon .

Expectation

It can be shown that \operatorname{E}\left[\varepsilon^T\Lambda\varepsilon\right]=\operatorname{tr}\left[\Lambda \Sigma\right] + \mu^T\Lambda\mu where \mu and \Sigma are the expected value and variance-covariance matrix of \varepsilon , respectively, and tr denotes the trace of a matrix. This result only depends on the existence of \mu and \Sigma ; in particular, normality of \varepsilon is not required.

A book treatment of the topic of quadratic forms in random variables is that of Mathai and Provost. Since the quadratic form is a scalar quantity, \varepsilon^T\Lambda\varepsilon = \operatorname{tr}(\varepsilon^T\Lambda\varepsilon) .

Next, by the cyclic property of the trace operator, \operatorname{E}[\operatorname{tr}(\varepsilon^T\Lambda\varepsilon)] = \operatorname{E}[\operatorname{tr}(\Lambda\varepsilon\varepsilon^T)]. Since the trace operator is a linear combination of the components of the matrix, it therefore follows from the linearity of the expectation operator that \operatorname{E}[\operatorname{tr}(\Lambda\varepsilon\varepsilon^T)] = \operatorname{tr}(\Lambda \operatorname{E}(\varepsilon\varepsilon^T)). A standard property of variances then tells us that this is \operatorname{tr}(\Lambda (\Sigma + \mu \mu^T)). Applying the cyclic property of the trace operator again, we get \operatorname{tr}(\Lambda\Sigma) + \operatorname{tr}(\Lambda \mu \mu^T) = \operatorname{tr}(\Lambda\Sigma) + \operatorname{tr}(\mu^T\Lambda\mu) = \operatorname{tr}(\Lambda\Sigma) + \mu^T\Lambda\mu.

Variance in the Gaussian case

In general, the variance of a quadratic form depends greatly on the distribution of \varepsilon . However, if \varepsilon does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that \Lambda is a symmetric matrix.

Then, \operatorname{var} \left[\varepsilon^T\Lambda\varepsilon\right] = 2\operatorname{tr}\left[\Lambda \Sigma\Lambda \Sigma\right] + 4\mu^T\Lambda\Sigma\Lambda\mu . In fact, this can be generalized to find the covariance between two quadratic forms on the same \varepsilon (once again, \Lambda_1 and \Lambda_2 must both be symmetric): \operatorname{cov}\left[\varepsilon^T\Lambda_1\varepsilon,\varepsilon^T\Lambda_2\varepsilon\right]=2\operatorname{tr}\left[\Lambda _1\Sigma\Lambda_2 \Sigma\right] + 4\mu^T\Lambda_1\Sigma\Lambda_2\mu .

In addition, a quadratic form such as this follows a generalized chi-squared distribution.

Computing the variance in the non-symmetric case

The case for general \Lambda can be derived by noting that \varepsilon^T\Lambda^T\varepsilon=\varepsilon^T\Lambda\varepsilon so \varepsilon^T\tilde{\Lambda}\varepsilon=\varepsilon^T\left(\Lambda+\Lambda^T\right)\varepsilon/2 is a quadratic form in the symmetric matrix \tilde{\Lambda}=\left(\Lambda+\Lambda^T\right)/2 , so the mean and variance expressions are the same, provided \Lambda is replaced by \tilde{\Lambda} therein.

Examples of quadratic forms

In the setting where one has a set of observations y and an operator matrix H , then the residual sum of squares can be written as a quadratic form in y : \textrm{RSS}=y^T(I-H)^T (I-H)y. For procedures where the matrix H is symmetric and idempotent, and the errors are Gaussian with covariance matrix \sigma^2I , \textrm{RSS}/\sigma^2 has a chi-squared distribution with k degrees of freedom and noncentrality parameter \lambda , where k=\operatorname{tr}\left[(I-H)^T(I-H)\right] \lambda=\mu^T(I-H)^T(I-H)\mu/2 may be found by matching the first two central moments of a noncentral chi-squared random variable to the expressions given in the first two sections. If Hy estimates \mu with no bias, then the noncentrality \lambda is zero and \textrm{RSS}/\sigma^2 follows a central chi-squared distribution.

See also

Quadratic form Covariance matrix Matrix representation of conic sections

Text derived from https://en.wikipedia.org/w/index.php?oldid=1372469701 under CC BY-SA 4.0; modifications have been made.

Words in each sectionExpectationExpectationExpectation: 223 words223Variance in the Gaussia…Variance in the Gaussian caseVariance in the Gaussian case: 144 words144Examples of quadratic f…Examples of quadratic formsExamples of quadratic forms: 141 words141Computing the variance …Computing the variance in the non-symmetric caseComputing the variance in the non-symmetric case: 68 words68See alsoSee alsoSee also: 30 words30
Words in each section. Measured from the stored article body; sections below the minimum are not plotted.

Provenance

snapshot enwiki-001372469701 · captured 2026-08-31 · ledger · version history

Cite this page as: encyclopegia.org, “Quadratic form (statistics)”, snapshot enwiki-001372469701. https://encyclopegia.org/a/quadratic-form-statistics

Sources cited in this revision

1 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.