Empirical likelihood
In probability theory and statistics, empirical likelihood (EL) is a nonparametric method for estimating the parameters of statistical models.
≈ 4 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In probability theory and statistics, empirical likelihood (EL) is a nonparametric method for estimating the parameters of statistical models. It requires fewer assumptions about the error distribution while retaining some of the merits in likelihood-based inference. The estimation method requires that the data are independent and identically distributed (iid).
It performs well even when the distribution is asymmetric or censored. EL methods can also handle constraints and prior information on parameters. Art Owen pioneered work in this area with his 1988 paper.
Given a set of n i.i.d. realizations y_i of random variables Y_i , then the empirical distribution function is \hat{F}(y):=\sum_{i=1}^n \pi_i I(Y_i , with the indicator function I and the (normalized) weights \pi_i . Then, the empirical likelihood is: L:=\prod_{i=1}^n \frac{\hat{F}(y_i)-\hat{F}(y_i-\delta y)}{\delta y}, where \delta y is a small number (potentially the difference to the next smaller sample). Empirical likelihood estimation can be augmented with side information by using further constraints (similar to the generalized estimating equations approach) for the empirical distribution function.
E.g. a constraint like the following can be incorporated using a Lagrange multiplier E[h(Y;\theta)]=\int_{-\infty}^\infty h(y;\theta) dF=0 which implies \hat{E}[h(y;\theta)]=\sum_{i=1}^n h(y_i;\theta)\pi_i=0 . With similar constraints, we could also model correlation. Given \ p_{i}:=\hat{F}(y_i)-\hat{F}(y_i-\delta y),\ i = 1,...,n such that p_i \geq 0 \text{ and } \sum_{i=1}^n\ p_{i} =1.
Then the empirical likelihood is again L(p_{1},...,p_{n})= \prod_{i=1}^n \ p_{i} . Using the Lagrangian multiplier method to maximize the logarithm of the empirical likelihood subject to the trivial normalization constraint, we find p_i = 1/n as a maximum. Therefore, \hat{F} is the empirical distribution function.
Estimation Procedure
EL estimates are calculated by maximizing the empirical likelihood function (see above) subject to constraints based on the estimating function and the trivial assumption that the probability weights of the likelihood function sum to 1. This procedure is represented as: \max_{\pi_{i}, \theta} \ln(L)= \max_{\pi_{i}, \theta} \sum_{i=1}^n \ln \pi_i subject to the constraints s.t. \sum_{i=1}^n\pi_i = 1, \sum_{i=1}^n\pi_i h(y_i;\theta) = 0,\forall i\in[1..n]\quad 0\le\pi_i. The value of the theta parameter can be found by solving the Lagrangian function \mathcal{L} = \sum_{i=1}^n \ln \pi_{i} + \mu (1- \sum_{i=1}^n \pi_{i})-n\tau' \sum_{i=1}^n \pi_{i} h(y_{i};\theta).
There is a clear analogy between this maximization problem and the one solved for maximum entropy. The parameters \pi_i are nuisance parameters. R(F)=L(F)/L(F_{n}) .
Consider sets of the form C = \{ T(F)| R(F) \geq r \} . Under such conditions a test of T(F)=t rejects when t does not belong to C , that is, when no distribution F with T(F)=t has likelihood L(F) \geq rL(F_{n}) .
The central result is for the mean of X. Clearly, some restrictions on F are needed, or else C = \reals^p whenever r . To see this, let: F = \epsilon \delta_{x} + (1- \epsilon) F_{n} If \epsilon is small enough and \epsilon >0 , then R(F) \geq r . But then, as x ranges through \reals^p , so does the mean of F , tracing out C = \reals^p . The problem can be solved by restricting to distributions F that are supported in a bounded set. It turns out to be possible to restrict attention t distributions with support in the sample, in other words, to distribution F \ll F_{n} .
Such method is convenient since the statistician might not be willing to specify a bounded support for F , and since t converts the construction of C into a finite dimensional problem.
See also
Bootstrapping (statistics) Jackknife (statistics) Nelson–Aalen estimator Survival_analysis#Discrete-time_survival_models
Text derived from https://en.wikipedia.org/w/index.php?oldid=1338845017 under CC BY-SA 4.0; modifications have been made.