Parametric statistics
Parametric statistics is a branch of statistics that is concerned with the analysis of and inference from data assuming that the underlying distribution, from which the observed data was drawn, can be described by a finite set of (unknown)…
≈ 5 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
Parametric statistics is a branch of statistics that is concerned with the analysis of and inference from data assuming that the underlying distribution, from which the observed data was drawn, can be described by a finite set of (unknown) parameters. In contrast, nonparametric statistics does not assume explicit (finite-parametric) mathematical forms for distributions when modeling data.
However, it may make some assumptions about that distribution, such as continuity or symmetry, or even an explicit mathematical shape but have a model for a distributional parameter that is not itself finite-parametric. Most well-known statistical methods are parametric.
Regarding nonparametric (and semiparametric) models, Sir David Cox has said, "These typically involve fewer assumptions of structure and distributional form but usually contain strong assumptions about independencies". In the typical setting of parameter estimation, the chosen parametric family of distributions has either a density function or a density mass function L_\theta , called the likelihood depending on the parameters \theta=(\theta_1,\dots,\theta_p) , and a finite sample of independent and identically distributed data X_1,\dots,X_n is given.
The goal is to estimate \theta (or a function thereof) based on the observed data. Method of moments (MoM): If the model parameters can be expressed as functions g_1,\dots,g_p of the moments of the distribution, then the moment estimates of the parameters are g_1\left(\sum_{i=1}^nX_i,\dots,\sum_{i=1}^nX_i^p\right),\dots,g_p\left(\sum_{i=1}^nX_i,\dots,\sum_{i=1}^n X_i^p\right) .
If the functions g_1,\dots,g_p are continuous, then the moment estimator is strongly consistent (due to the strong law of large numbers). If the functions g_1,\dots,g_p are even differentiable, then the moment estimator is also asymptotically normal (due to the central limit theorem and the delta method).
Least square estimation (LSE): This method applies to a regression setting, where the data arises in pairs (X_1,Y_1),\dots,(X_n,Y_n) and a regression function f is to be determined. The model parameters are chosen such that the sum of squared differences \sum_{i=1}^n(Y_i-f_\theta(X_i))^2 between the observed data and the model prediction is minimal.
In fact, LSE is a special case of MLE, where the conditional distribution of Y given X is normally distributed. A best linear unbiased estimator (BLUE) has minimum variance among all unbiased estimators \hat\theta that are linear in the data, that is, \hat\theta=\sum_{i=1}^n\alpha_i X_i, for some coefficients \alpha_1,\dots,\alpha_n\in\R . In a sense, BLUEs are UMVU estimators with the additional restriction of being linear.
In the regression setting, the Gauss–Markov theorem states that the least squares estimator is the unique BLUE. This theorem also shows (considering the special case of constant regression) that the empirical mean is a BLUE for estimating the mean.
In regular statistical models, it can be shown that the variance of unbiased estimators cannot be arbitrarily small: Any unbiased estimator T of the quantity q(\theta) is bounded from below by the universal bound \mathrm{Var}_\theta[T(X)]\geq \nabla q(\theta)I(\theta)^{-1} \nabla q(\theta)^T, where I(\theta) is the Fisher-information matrix of the statistical model. The right hand-side is the so-called Cramér–Rao bound.
Estimators for which the variance equals the Cramér–Rao bound are called efficient and are almost UMVU, since they minimize the variance over all regular unbiased estimators (there might be non-regular estimators with smaller variance). On the contrary, not every UMVUE is efficient.
In fact, an estimator T is efficient if and only if (i) the statistical model is an exponential family and (ii) T is the natural sufficient statistic.
Example
The normal family of distributions all have the same general shape and are parameterized by mean and standard deviation. That means that if the mean and standard deviation are known and if the distribution is normal, the probability of any future observation lying in a given range is known. Suppose that we have a sample of 99 test scores with a mean of 100 and a standard deviation of 1.
If we assume all 99 test scores are random observations from a normal distribution, then we predict there is a 1% chance that the 100th test score will be higher than 102.33 (that is, the mean plus 2.33 standard deviations), assuming that the 100th test score comes from the same distribution as the others. Parametric statistical methods are used to compute the 2.33 value above, given 99 independent observations from the same normal distribution.
A non-parametric estimate of the same thing is the maximum of the first 99 scores. We don't need to assume anything about the distribution of test scores to reason that before we gave the test it was equally likely that the highest score would be any of the first 100. Thus there is a 1% chance that the 100th score is higher than any of the 99 that preceded it.
See also
Aggregated distribution All models are wrong Inverse problem Parametric model Sufficient statistic
Text derived from https://en.wikipedia.org/w/index.php?oldid=1359528627 under CC BY-SA 4.0; modifications have been made.