{
  "id": "enwiki-001358942527",
  "slug": "robust-regression",
  "title": "Robust regression",
  "qid": "",
  "lead": "In robust statistics, robust regression seeks to overcome some limitations of traditional regression analysis. A regression analysis models the relationship between one or more independent variables and a dependent variable. Standard types of regression, such as ordinary least squares, have favourable properties if their underlying assumptions are true, but can give misleading results otherwise (i.e. are not robust to assumption violations). Robust regression methods are designed to limit the effect that violations of assumptions by the underlying data-generating process have on regression estimates. For example, least squares estimates for regression models are highly sensitive to outliers: an outlier with twice the error magnitude of a typical observation contributes four (two squared) times as much to the squared error loss, and therefore has more leverage over the regression estimates. The Huber loss function is a robust alternative to standard square error loss that reduces outliers' contributions to the squared error loss, thereby limiting their impact on regression estimates.",
  "body": "In robust statistics, robust regression seeks to overcome some limitations of traditional regression analysis. A regression analysis models the relationship between one or more independent variables and a dependent variable. Standard types of regression, such as ordinary least squares, have favourable properties if their underlying assumptions are true, but can give misleading results otherwise (i.e. are not robust to assumption violations). Robust regression methods are designed to limit the effect that violations of assumptions by the underlying data-generating process have on regression estimates. For example, least squares estimates for regression models are highly sensitive to outliers: an outlier with twice the error magnitude of a typical observation contributes four (two squared) times as much to the squared error loss, and therefore has more leverage over the regression estimates. The Huber loss function is a robust alternative to standard square error loss that reduces outliers' contributions to the squared error loss, thereby limiting their impact on regression estimates. Another robust method is the use of unit weights (Wainer & Thissen, 1976), a method that can be applied when there are multiple predictors of a single outcome. Ernest Burgess (1928) used unit weights to predict success on parole. He scored 21 positive factors as present (e.g., \"no prior arrest\" = 1) or absent (\"prior arrest\" = 0), then summed to yield a predictor score, which was shown to be a useful predictor of parole success. Samuel S. Wilks (1938) showed that nearly all sets of regression weights sum to composites that are very highly correlated with one another, including unit weights, a result sometimes referred to as Wilks' theorem (Ree, Carretta, & Earles, 1998). Robyn Dawes (1979) examined decision making in applied settings, showing that simple models with unit weights often outperformed human experts. Bobko, Roth, and Buster (2007) reviewed the literature on unit weights and concluded that decades of empirical studies show that unit weights perform similar to ordinary regression weights on cross validation. Burgess, E. W. (1928). \"Factors determining success or failure on parole\". In A. A. Bruce (Ed.), The Workings of the Indeterminate Sentence Law and Parole in Illinois (pp.&nbsp;205–249). Springfield, Illinois: Illinois State Parole Board. Google books Dawes, Robyn M. (1979). \"The robust beauty of improper linear models in decision making\". American Psychologist, volume 34, pages 571-582. . archived pdf Lerman, G.; McCoy, M.; Tropp, J. A.; Zhang T. (2012). \"Robust computation of linear models, or how to find a needle in a haystack\" , . Ree, M. J., Carretta, T. R., & Earles, J. A. (1998). \"In top-down decisions, weighting variables does not matter: A consequence of Wilk's theorem. Organizational Research Methods, volume 1(4), pages 407-420. Wainer, H., & Thissen, D. (1976). \"Three steps toward robust regression.\" Psychometrika, volume 41(1), pages 9–34. Wilks, S. S. (1938). \"Weighting systems for linear functions of correlated variables when there is no dependent variable\". Psychometrika, volume 3, pages 23–40.",
  "categories": [
    "22896689",
    "22929658",
    "23603337",
    "24623845",
    "24745122",
    "26220375",
    "27424729",
    "37160764",
    "80930309"
  ],
  "license": "CC BY-SA 4.0",
  "source_uri": "https://en.wikipedia.org/w/index.php?oldid=1358942527",
  "content_hash": "",
  "created_at": null,
  "updated_at": 1781216557.0,
  "claims": [],
  "contradictions": [],
  "confidence": 0.6,
  "csl": {
    "id": "enwiki-001358942527",
    "type": "webpage",
    "title": "Robust regression",
    "author": [
      {
        "family": "Anticloud FZ LLE",
        "literal": "Anticloud FZ LLE"
      }
    ],
    "container-title": "encyclopegia.org",
    "publisher": "Anticloud FZ LLE",
    "publisher-place": "Dubai, United Arab Emirates",
    "URL": "https://encyclopegia.org/a/robust-regression",
    "issued": {
      "date-parts": [
        [
          2026,
          11,
          16
        ]
      ]
    },
    "accessed": {
      "date-parts": [
        [
          2026,
          11,
          16
        ]
      ]
    },
    "license": "CC BY-SA 4.0",
    "DOI": "eo1:enwiki-001358942527",
    "isBasedOn": "https://en.wikipedia.org/w/index.php?oldid=1358942527",
    "note": "Immutable snapshot enwiki-001358942527"
  }
}