{
  "id": "enwiki-001317993932",
  "slug": "smooth-maximum",
  "title": "Smooth maximum",
  "qid": "",
  "lead": "In mathematics, a smooth maximum of an indexed family x 1 , ..., x n of numbers is a smooth approximation to the maximum function \\max(x_1,\\ldots,x_n), meaning a parametric family of functions m_\\alpha(x_1,\\ldots,x_n) such that for every , the function is smooth, and the family converges to the maximum function as . The concept of smooth minimum is similarly defined. In many cases, a single family approximates both: maximum as the parameter goes to positive infinity, minimum as the parameter goes to negative infinity; in symbols, as and as . The term can also be used loosely for a specific smooth function that behaves similarly to a maximum, without necessarily being part of a parametrized family.",
  "body": "In mathematics, a smooth maximum of an indexed family x 1 ,&nbsp;...,&nbsp;x n of numbers is a smooth approximation to the maximum function \\max(x_1,\\ldots,x_n), meaning a parametric family of functions m_\\alpha(x_1,\\ldots,x_n) such that for every , the function is smooth, and the family converges to the maximum function as . The concept of smooth minimum is similarly defined. In many cases, a single family approximates both: maximum as the parameter goes to positive infinity, minimum as the parameter goes to negative infinity; in symbols, as and as . The term can also be used loosely for a specific smooth function that behaves similarly to a maximum, without necessarily being part of a parametrized family. \\mathcal{S}_\\alpha (x_1,\\ldots,x_n) = \\frac{\\sum_{i=1}^n x_i e^{\\alpha x_i}}{\\sum_{i=1}^n e^{\\alpha x_i}} \\mathcal{S}_\\alpha has the following properties: \\mathcal{S}_\\alpha\\to \\max as \\alpha\\to\\infty \\mathcal{S}_0 is the arithmetic mean of its inputs \\mathcal{S}_\\alpha\\to \\min as \\alpha\\to -\\infty The gradient of \\mathcal{S}_{\\alpha} is closely related to softmax and is given by \\nabla_{x_i}\\mathcal{S}_\\alpha (x_1,\\ldots,x_n) = \\frac{e^{\\alpha x_i}}{\\sum_{j=1}^n e^{\\alpha x_j}} [1 + \\alpha(x_i - \\mathcal{S}_\\alpha (x_1,\\ldots,x_n))]. This makes the softmax function useful for optimization techniques that use gradient descent. This operator is sometimes called the Boltzmann operator, after the Boltzmann distribution. \\mathrm{LSE}_\\alpha(x_1, \\ldots, x_n) = \\frac{1}{\\alpha} \\log \\sum_{i=1}^n \\exp \\alpha x_i This can also be normalized if the x_i are all non-negative, yielding a function with domain [0,\\infty)^n and range [0, \\infty) : g(x_1, \\ldots, x_n) = \\log \\left( \\sum_{i=1}^n \\exp x_i - (n-1) \\right) The (n - 1) term corrects for the fact that \\exp(0) = 1 by canceling out all but one zero exponential, and \\log 1 = 0 if all x_i are zero. \\mathrm{mm}_\\alpha(x) = \\frac{1}{\\alpha} \\log \\frac{1}{n} \\sum_{i=1}^n \\exp \\alpha x_i It is a non-expansive operator. As \\alpha \\to \\infty , it acts like a maximum. As \\alpha \\to 0 , it acts like an arithmetic mean. As \\alpha \\to -\\infty , it acts like a minimum. This operator can be viewed as a particular instantiation of the quasi-arithmetic mean. It can also be derived from information theoretical principles as a way of regularizing policies with a cost function defined by KL divergence. The operator has previously been utilized in other areas, such as power engineering. \\| (x_1, \\ldots, x_n) \\|_p = \\left( \\sum_{i=1}^n |x_i|^p \\right)^\\frac{1}{p} which converges to \\| (x_1, \\ldots, x_n) \\|_\\infty = \\max_{1\\leq i\\leq n} |x_i| as p \\to \\infty . An advantage of the p-norm is that it is a norm. As such it is scale invariant (homogeneous): \\| (\\lambda x_1, \\ldots, \\lambda x_n) \\|_p = |\\lambda| \\cdot \\| (x_1, \\ldots, x_n) \\|_p , and it satisfies the triangle inequality. &= \\frac{a + b + |a - b|_\\varepsilon}{2} \\\\ &= \\frac{a + b + \\sqrt{(a - b)^2 + \\varepsilon}}{2} \\end{align} where \\varepsilon \\geq 0 is a parameter. As \\varepsilon \\to 0 , |\\cdot|_\\varepsilon \\to |\\cdot| and thus \\textstyle\\max_\\varepsilon \\to \\max . ==See also== LogSumExp Softmax function Generalized mean ==References== https://www.johndcook.com/soft_maximum.pdf M. Lange, D. Zühlke, O. Holz, and T. Villmann, \"Applications of lp-norms and their smooth approximations for gradient based learning vector quantization,\" in Proc. ESANN, Apr. 2014, pp. 271-276. (https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2014-153.pdf)",
  "categories": [
    "23118434",
    "23603337",
    "24185365",
    "29176186"
  ],
  "license": "CC BY-SA 4.0",
  "source_uri": "https://en.wikipedia.org/w/index.php?oldid=1317993932",
  "content_hash": "",
  "created_at": null,
  "updated_at": 1761026614.0,
  "claims": [],
  "contradictions": [],
  "confidence": 0.6,
  "csl": {
    "id": "enwiki-001317993932",
    "type": "webpage",
    "title": "Smooth maximum",
    "author": [
      {
        "family": "Anticloud FZ LLE",
        "literal": "Anticloud FZ LLE"
      }
    ],
    "container-title": "encyclopegia.org",
    "publisher": "Anticloud FZ LLE",
    "publisher-place": "Dubai, United Arab Emirates",
    "URL": "https://encyclopegia.org/a/smooth-maximum",
    "issued": {
      "date-parts": [
        [
          2026,
          11,
          20
        ]
      ]
    },
    "accessed": {
      "date-parts": [
        [
          2026,
          11,
          20
        ]
      ]
    },
    "license": "CC BY-SA 4.0",
    "DOI": "eo1:enwiki-001317993932",
    "isBasedOn": "https://en.wikipedia.org/w/index.php?oldid=1317993932",
    "note": "Immutable snapshot enwiki-001317993932"
  }
}