Completeness (statistics)
≈ 6 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In statistics, completeness is a property of a statistic computed on a sample dataset in relation to a parametric model of the dataset. It is opposed to the concept of an ancillary statistic. While an ancillary statistic contains no information about the model parameters, a complete statistic contains only information about the parameters, and no ancillary information. It is closely related to the concept of a sufficient statistic which contains all of the information that the dataset provides about the parameters.
In statistics, completeness is a property of a statistic computed on a sample dataset in relation to a parametric model of the dataset. It is opposed to the concept of an ancillary statistic. While an ancillary statistic contains no information about the model parameters, a complete statistic contains only information about the parameters, and no ancillary information. It is closely related to the concept of a sufficient statistic which contains all of the information that the dataset provides about the parameters.
Definition
Consider a random variable X whose probability distribution belongs to a parametric model P θ parametrized by θ. Say T is a statistic; that is, the composition of a measurable function with a random sample X 1 ,...,X n . The statistic T is said to be complete for the distribution of X if, for every measurable function g, \text{if }\operatorname{E}_\theta(g(T))=0\text{ for all }\theta\text{ then }\mathbf{P}_\theta(g(T)=0)=1\text{ for all }\theta.
The statistic T is said to be boundedly complete for the distribution of X if this implication holds for every measurable function g that is also bounded. The Bernoulli model admits a complete statistic. Let X be a random sample of size n such that each X i has the same Bernoulli distribution with parameter p. Let T be the number of 1s observed in the sample, i.e. \textstyle T = \sum_{i=1}^n X_i . T is a statistic of X which has a binomial distribution with parameters (n,p). If the parameter space for p is (0,1), then T is a complete statistic.
To see this, note that \operatorname{E}_p(g(T)) = \sum_{t=0}^n {g(t){n \choose t}p^{t}(1-p)^{n-t}} = (1-p)^n \sum_{t=0}^n {g(t){n \choose t}\left(\frac{p}{1-p}\right)^t} . Observe also that neither p nor 1 − p can be 0. Hence E_p(g(T)) = 0 if and only if: \sum_{t=0}^n g(t){n \choose t}\left(\frac{p}{1-p}\right)^t = 0. On denoting p/(1 − p) by r, one gets: \sum_{t=0}^n g(t){n \choose t}r^t = 0 . First, observe that the range of r is the positive reals.
Also, E(g(T)) is a polynomial in r and, therefore, can only be identical to 0 if all coefficients are 0, that is, g(t) = 0 for all t. It is important to notice that the result that all coefficients must be 0 was obtained because of the range of r. Had the parameter space been finite and with a number of elements less than or equal to n, it might be possible to solve the linear equations in g(t) obtained by substituting the values of r and get solutions different from 0.
For example, if n = 1 and the parameter space is {0.5}, a single observation and a single parameter value, T is not complete. Observe that, with the definition: g(t) = 2(t-0.5), \, then, E(g(T)) = 0 although g(t) is not 0 for t = 0 nor for t = 1. s((X_1, X_2)) = X_1 + X_2 is a complete statistic for θ. To show this, it is sufficient to demonstrate that there is no non-zero function g such that the expectation of g(s(X_1, X_2)) = g(X_1+X_2) remains zero regardless of the value of θ. That fact may be seen as follows.
The probability distribution of X 1 + X 2 is normal with expectation 2θ and variance 2. Its probability density function in x is therefore proportional to \exp\left(-(x-2\theta)^2/4\right). The expectation of g above would therefore be a constant times \int_{-\infty}^\infty g(x)\exp\left(-(x-2\theta)^2/4\right)\,dx. A bit of algebra reduces this to k(\theta) \int_{-\infty}^\infty h(x)e^{x\theta}\,dx, where k(θ) is nowhere zero and h(x)=g(x)e^{-x^2/4}.
As a function of θ this is a two-sided Laplace transform of h, and cannot be identically zero unless h is zero almost everywhere. The exponential is not zero, so this can only happen if g is zero almost everywhere. By contrast, the statistic (X_1,X_2) is sufficient but not complete. It admits a non-zero unbiased estimator of zero, namely X_1-X_2 . \mathbb E \left[ \frac {X_{(n)}} {X_{(1)} } \right] - m = 0 We have thus shown that there exists a function g\left(X_{(1)}, X_{(n)}\right) which is not 0 everywhere but which has expectation 0 .
The pair is thus not complete.
Importance of completeness
The notion of completeness has many applications in statistics, particularly in the following theorems of mathematical statistics.
Lehmann–Scheffé theorem
Completeness occurs in the Lehmann–Scheffé theorem, which states that given a complete and sufficient statistic T , an unbiased estimator for the parameter θ of the form g(T) , for some measurable function g , is the best mean-unbiased estimator for θ. In other words, the estimator g(T) has the smallest expected loss for any convex loss function; in many practical applications with the squared loss-function, it has a smaller mean squared error among any estimators with the same expected value. There are examples where the minimal sufficient statistic is not complete and where several alternative statistics for unbiased estimation of θ exist, while some of them have lower variance than others. See also minimum-variance unbiased estimator.
Basu's theorem
Bounded completeness occurs in Basu's theorem, which states that a statistic that is both boundedly complete and sufficient is independent of any ancillary statistic.
Bahadur's theorem
Bounded completeness also occurs in Bahadur's theorem. In the case where there exists at least one minimal sufficient statistic, a statistic which is sufficient and boundedly complete, is necessarily minimal sufficient.
Notes
Text derived from https://en.wikipedia.org/w/index.php?oldid=1361576191 under CC BY-SA 4.0; modifications have been made.