Consistent estimator

In statistics, a consistent estimator or asymptotically consistent estimator is an estimator—a rule for computing estimates of a parameter θ 0 —having the property that as the number of data points used increases indefinitely, the resulting sequence of estimates converges in probability to θ 0 . This means that the distributions of the estimates become more and more concentrated near the true value of the parameter being estimated, so that the probability of the estimator being arbitrarily close to θ 0 converges to one. In practice one constructs an estimator as a function of an available sample of size n, and then imagines being able to keep collecting data and expanding the sample ad infinitum. In this way one would obtain a sequence of estimates indexed by n, and consistency is a property of what occurs as the sample size “grows to infinity”. If the sequence of estimates can be mathematically shown to converge in probability to the true value θ 0 , it is called a consistent estimator; otherwise the estimator is said to be inconsistent. Consistency as defined here is sometimes referred to as weak consistency. When we replace convergence in probability with almost sure convergence, then the estimator is said to be strongly consistent. Consistency is related to bias; see bias versus consistency.

In statistics, a consistent estimator or asymptotically consistent estimator is an estimator—a rule for computing estimates of a parameter θ 0 —having the property that as the number of data points used increases indefinitely, the resulting sequence of estimates converges in probability to θ 0 . This means that the distributions of the estimates become more and more concentrated near the true value of the parameter being estimated, so that the probability of the estimator being arbitrarily close to θ 0 converges to one. In practice one constructs an estimator as a function of an available sample of size n, and then imagines being able to keep collecting data and expanding the sample ad infinitum. In this way one would obtain a sequence of estimates indexed by n, and consistency is a property of what occurs as the sample size “grows to infinity”. If the sequence of estimates can be mathematically shown to converge in probability to the true value θ 0 , it is called a consistent estimator; otherwise the estimator is said to be inconsistent. Consistency as defined here is sometimes referred to as weak consistency. When we replace convergence in probability with almost sure convergence, then the estimator is said to be strongly consistent. Consistency is related to bias; see bias versus consistency. \underset{n\to\infty}{\operatorname{plim}}\;T_n = \theta. i.e. if, for all ε > 0 \lim_{n\to\infty}\Pr\big(|T_n-\theta| > \varepsilon\big) = 0. An estimator T n of parameter θ is said to be strongly consistent, if it converges almost surely to the true value of the parameter: \Pr\big(\lim_{n\to\infty}T_n = \theta\big) = 1. A more rigorous definition takes into account the fact that θ is actually unknown, and thus, the convergence in probability must take place for every possible value of this parameter. Suppose is a family of distributions (the parametric model), and is an infinite sample from the distribution p θ . Let { T n (X θ ) } be a sequence of estimators for some parameter g(θ). Usually, T n will be based on the first n observations of a sample. Then this sequence {T n } is said to be (weakly) consistent if \underset{n\to\infty}{\operatorname{plim}}\;T_n(X^{\theta}) = g(\theta),\ \ \text{for all}\ \theta\in\Theta. This definition uses g(θ) instead of simply θ, because often one is interested in estimating a certain function or a sub-vector of the underlying parameter. In the next example, we estimate the location parameter of the model, but not the scale: Suppose one has a sequence of statistically independent observations {X 1 , X 2 , ...} from a normal N(μ, σ 2 ) distribution. To estimate μ based on the first n observations, one can use the sample mean: T n  = (X 1 + ... + X n )/n. This defines a sequence of estimators, indexed by the sample size n. From the properties of the normal distribution, we know the sampling distribution of this statistic: T n is itself normally distributed, with mean μ and variance σ 2 /n. Equivalently, \scriptstyle (T_n-\mu)/(\sigma/\sqrt{n}) has a standard normal distribution: \Pr\!\left[\,|T_n-\mu|\geq\varepsilon\,\right] = \Pr\!\left[ \frac{\sqrt{n}\,\big|T_n-\mu\big|}{\sigma} \geq \sqrt{n}\varepsilon/\sigma \right] = 2\left(1-\Phi\left(\frac{\sqrt{n}\,\varepsilon}{\sigma}\right)\right) \to 0 as n tends to infinity, for any fixed . Therefore, the sequence T n of sample means is consistent for the population mean μ (recalling that \Phi is the cumulative distribution of the standard normal distribution). Slutsky's theorem can be used to combine several different estimators, or an estimator with a non-random convergent sequence. If T n  → d α, and S n  → p β, then \begin{align} & T_n + S_n \ \xrightarrow{d}\ \alpha+\beta, \\ & T_n S_n \ \xrightarrow{d}\ \alpha \beta, \\ & T_n / S_n \ \xrightarrow{d}\ \alpha/\beta, \text{ provided that }\beta\neq0 \end{align} If estimator T n is given by an explicit formula, then most likely the formula will employ sums of random variables, and then the law of large numbers can be used: for a sequence {X n } of random variables and under suitable conditions, \frac{1}{n}\sum_{i=1}^n g(X_i) \ \xrightarrow{p}\ \operatorname{E}[\,g(X)\,] If estimator T n is defined implicitly, for example as a value that maximizes certain objective function (see extremum estimator), then a more complicated argument involving stochastic equicontinuity has to be used. An estimator can be unbiased but not consistent. For example, for an iid sample {x ,..., x } one can use T (X) = x as the estimator of the mean E[X]. Note that here the sampling distribution of T is the same as the underlying distribution (for any n, as it ignores all points but the last). So E[T (X)] = E[X] for any n, hence it is unbiased, but it does not converge to any value. However, if a sequence of estimators is unbiased and converges to a value, then it is consistent, as it must converge to the correct value. \Pr(T_n) = \begin{cases} 1 - 1/n, & \mbox{if }\, T_n = \theta \\ 1/n, & \mbox{if }\, T_n = n\delta + \theta \end{cases} We can see that T_n \xrightarrow{p} \theta , \operatorname{E}[T_n] = \theta + \delta , and the bias does not converge to zero.

Text derived from https://en.wikipedia.org/w/index.php?oldid=1333808451 under CC BY-SA 4.0; modifications have been made.