elastic net regularization

Uses your system voice; voice and rate in settings.

Q5353562Snapshot

≈ 3 min read

Names in other languages

  • УкраїнськаЕластично-сіткова регуляризація
  • বাংলাইলাস্টিক নেট রেগুলারাইজেশন
  • Englishelastic net regularization
  • eoelasta reta reguligo
  • فارسیمتعادل‌سازی کشسان
  • 日本語エラスティックネット

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In statistics and, in particular, in the fitting of linear or logistic regression models, the elastic net is a regularized regression method that linearly combines the L 1 and L 2 penalties of the lasso and ridge methods. Nevertheless, elastic net regularization is typically more accurate than both methods with regard to reconstruction.

In statistics and, in particular, in the fitting of linear or logistic regression models, the elastic net is a regularized regression method that linearly combines the L 1 and L 2 penalties of the lasso and ridge methods. Nevertheless, elastic net regularization is typically more accurate than both methods with regard to reconstruction.

Specification

The elastic net method overcomes the limitations of the LASSO (least absolute shrinkage and selection operator) method which uses a penalty function based on \|\beta\|_1 = \textstyle \sum_{j=1}^p |\beta_j|. Use of this penalty function has several limitations. For example, in the "large p, small n" case (high-dimensional data with few examples), the LASSO selects at most n variables before it saturates. Also if there is a group of highly correlated variables, then the LASSO tends to select one variable from a group and ignore the others.

To overcome these limitations, the elastic net adds a quadratic part ( \|\beta\|^2 ) to the penalty, which when used alone is ridge regression (known also as Tikhonov regularization). The estimates from the elastic net method are defined by \hat{\beta} \equiv \underset{\beta}{\operatorname{argmin}} (\| y-X \beta \|^2 + \lambda_2 \|\beta\|^2 + \lambda_1 \|\beta\|_1) . The quadratic penalty term makes the loss function strongly convex, and it therefore has a unique minimum.

The elastic net method includes the LASSO and ridge regression: in other words, each of them is a special case where \lambda_1 = \lambda, \lambda_2 = 0 or \lambda_1 = 0, \lambda_2 = \lambda . Meanwhile, the naive version of elastic net method finds an estimator in a two-stage procedure : first for each fixed \lambda_2 it finds the ridge regression coefficients, and then does a LASSO type shrinkage. This kind of estimation incurs a double amount of shrinkage, which leads to increased bias and poor predictions.

To improve the prediction performance, sometimes the coefficients of the naive version of elastic net is rescaled by multiplying the estimated coefficients by (1 + \lambda_2) .

Examples of where the elastic net method has been applied are: Support vector machine Metric learning Portfolio optimization Cancer prognosis function β=SVEN(X, y, t, λ2); [n,p] = size(X); X2 = [bsxfun(@minus, X, y./t); bsxfun(@plus, X, y./t)]'; Y2 = [ones(p,1);-ones(p,1)]; if 2p > n then w = SVMPrimal(X2, Y2, C = 1/(2*λ2)); α = C * max(1-Y2.*(X2*w), 0); else α = SVMDual(X2, Y2, C = 1/(2*λ2)); end if β = t * (α(1:p) - α(p+1:2p)) / sum(α);

Text derived from https://en.wikipedia.org/w/index.php?oldid=1374244169 under CC BY-SA 4.0; modifications have been made.

Provenance

snapshot elastic-net-regularization-609673-63aaa6ad9469d8f2 · hash 63aaa6ad9469d8f27285bfbc899b57590eb1ea0244e88ce7bfe1acee17a565ba · captured 1789061922.0 · version history

Cite this page as: encyclopegia.org, “elastic net regularization”, snapshot elastic-net-regularization-609673-63aaa6ad9469d8f2. https://encyclopegia.org/a/elastic-net-regularization

Sources cited in this revision

No external citation was extracted for this revision. That is a statement about the source, not a claim that none exists.