Logistic function

Uses your system voice; voice and rate in settings.

Snapshot

≈ 21 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

A logistic function or logistic curve is a common S-shaped curve (sigmoid curve) with the equation f(x) = \frac{L}{1 + e^{-k(x-x_0)}} where * L is the carrying capacity, the supremum of the values of the function; * k is the logistic growth rate, the steepness of the curve; and * x_0 is the x value of the function's midpoint. The logistic function has domain the real numbers, the limit as x tends to -\infty is 0, and the limit as x tends to +\infty is L . The exponential function with negated argument is used to define the standard logistic function where L=1, k=1, x_0=0 , which has the equation f(x) = \frac{1}{1 + e^{-x}} and is sometimes simply called the sigmoid function. It is also sometimes called the expit, being the inverse function of the logit. The logistic function finds applications in a range of fields, including biology (especially ecology), biomathematics, chemistry, demography, economics, geoscience, mathematical psychology, probability, sociology, political science, linguistics, statistics, and artificial neural networks. There are various generalizations, depending on the field.

A logistic function or logistic curve is a common S-shaped curve (sigmoid curve) with the equation f(x) = \frac{L}{1 + e^{-k(x-x_0)}} where L is the carrying capacity, the supremum of the values of the function; k is the logistic growth rate, the steepness of the curve; and x_0 is the x value of the function's midpoint. The logistic function has domain the real numbers, the limit as x tends to -\infty is 0, and the limit as x tends to +\infty is L . The exponential function with negated argument is used to define the standard logistic function where L=1, k=1, x_0=0 , which has the equation f(x) = \frac{1}{1 + e^{-x}} and is sometimes simply called the sigmoid function. It is also sometimes called the expit, being the inverse function of the logit. The logistic function finds applications in a range of fields, including biology (especially ecology), biomathematics, chemistry, demography, economics, geoscience, mathematical psychology, probability, sociology, political science, linguistics, statistics, and artificial neural networks. There are various generalizations, depending on the field. The is the logistic function with parameters k = 1 , x_0 = 0 , L = 1 , which yields f(x) = \frac{1}{1 + e^{-x}} = \frac{e^x}{e^x + 1} = \frac{e^{x/2}}{e^{x/2} + e^{-x/2}}. In practice, due to the nature of the exponential function e^{-x} , it is often sufficient to compute the standard logistic function for x over a small range of real numbers, such as a range contained in [−6, +6], as it quickly converges very close to its saturation values of 0 and 1.

Symmetries

The logistic function has the symmetry property that 1 - f(x) = f(-x). This reflects that the growth from 0 when x is small is symmetric with the decay of the gap to the limit (1) when x is large. Further, x \mapsto f(x) - 1/2 is an odd function. The sum of the logistic function and its reflection about the vertical axis, f(-x) , is \frac{1}{1 + e^{-x}} + \frac{1}{1 + e^{-(-x)}} = \frac{e^x}{e^x + 1} + \frac{1}{e^x + 1} = 1. The logistic function is thus rotationally symmetrical about the point (0, 1/2).

Inverse function

The logistic function is the inverse of the natural logit function \operatorname{logit} p = \log \frac p {1-p} \quad \text{ for }\, 0 and so converts the logarithm of odds into a probability. {{Hidden| Proof| \operatorname{logistic}(\operatorname{logit}(p)) = \dfrac{1}{1+e^{-\log\left(\frac{p}{1-p}\right)}} = \dfrac{1}{1+e^{\log\left(\left(\frac{p}{1-p}\right)^{-1}\right)}} = \dfrac{1}{1+\frac{1-p}{p}} = \dfrac{p}{p+1-p} = p }} The conversion from the log-likelihood ratio of two alternatives also takes the form of a logistic curve.

Hyperbolic tangent

The logistic function is an offset and scaled hyperbolic tangent function: f(x) = \frac12 + \frac12 \tanh\left(\frac{x}{2}\right), or \tanh(x) = 2 f(2x) - 1. This follows from \begin{align} \tanh(x) & = \frac{e^x - e^{-x}}{e^x + e^{-x}} \\ &= \frac{e^x \cdot \left(1 - e^{-2x}\right)}{e^x \cdot \left(1 + e^{-2x}\right)} \\ &= f(2x) - \frac{e^{-2x}}{1 + e^{-2x}} \\ &= f(2x) - \frac{e^{-2x} + 1 - 1}{1 + e^{-2x}} \\ &= 2f(2x) - 1. \end{align} The hyperbolic-tangent relationship leads to another form for the logistic function's derivative: \frac{d}{dx} f(x) = \frac14 \operatorname{sech}^2\left(\frac{x}{2}\right), which ties the logistic function into the logistic distribution. Geometrically, the hyperbolic tangent function is the hyperbolic angle on the unit hyperbola x^2 - y^2 = 1 , which factors as (x + y)(x - y) = 1 , and thus has asymptotes the lines through the origin with slope and with slope , and vertex at corresponding to the range and midpoint ( ) of tanh. Analogously, the logistic function can be viewed as the hyperbolic angle on the hyperbola xy - y^2 = 1 , which factors as y(x - y) = 1 , and thus has asymptotes the lines through the origin with slope and with slope , and vertex at , corresponding to the range and midpoint ( ) of the logistic function. Parametrically, hyperbolic cosine and hyperbolic sine give coordinates on the unit hyperbola: \left( (e^t + e^{-t})/2, (e^t - e^{-t})/2\right) , with quotient the hyperbolic tangent. Similarly, \bigl(e^{t/2} + e^{-t/2}, e^{t/2}\bigr) parametrizes the hyperbola xy - y^2 = 1 , with quotient the logistic function. These correspond to linear transformations (and rescaling the parametrization) of the hyperbola xy = 1 , with parametrization (e^{-t}, e^t) : the parametrization of the hyperbola for the logistic function corresponds to t/2 and the linear transformation \bigl( \begin{smallmatrix} 1 & 1\\ 0 & 1 \end{smallmatrix} \bigr) , while the parametrization of the unit hyperbola (for the hyperbolic tangent) corresponds to the linear transformation \tfrac{1}{2}\bigl( \begin{smallmatrix} 1 & 1\\ -1 & 1 \end{smallmatrix} \bigr) . f(x) = \frac{1}{1 + e^{-x}} = \frac{e^x}{1 + e^x}, \begin{align} \frac{d}{dx} f(x) &= \frac{e^x \cdot (1 + e^x) - e^x \cdot e^x}{{\left(1 + e^x\right)}^2} \\[1ex] &= \frac{e^x}{{\left(1 + e^x\right)}^2} \\[1ex] &= \frac{e^x}{1 + e^x} \cdot \frac{1}{1 + e^x} \\[1ex] &= \frac{e^x}{1 + e^x} \left(\frac{1 + e^x - e^x}{1 + e^x}\right) \\[1.2ex] &= \frac{e^x}{1 + e^x} \left(1-\frac{e^x}{1 + e^x}\right) \\[1.2ex] &= f(x)\left(1 - f(x)\right) \end{align} from which all higher derivatives can be derived algebraically. For example, The logistic distribution is a location–scale family, which corresponds to parameters of the logistic function. If is fixed, then the midpoint is the location and the slope is the scale. Conversely, its antiderivative can be computed by the substitution u = 1 + e^x , since f(x) = \frac{e^x}{1 + e^x} = \frac{u'}{u}, so (dropping the constant of integration) \int \frac{e^x}{1 + e^x}\,dx = \int \frac{1}{u}\,du = \ln u = \ln (1 + e^x). In artificial neural networks, this is known as the softplus function and (with scaling) is a smooth approximation of the ramp function, just as the logistic function (with scaling) is a smooth approximation of the Heaviside step function. The standard logistic function is analytic on the whole real line since f : \mathbb{R} \to \mathbb{R} , f(x) = \frac{1}{1+e^{-x}} = h(g(x)) where g : \mathbb{R} \to \mathbb{R} , g(x) = 1 + e^{-x} and h : (0, \infty) \to (0, \infty) , h(x) = \frac{1}{x} are analytic on their domains, and the composition of analytic functions is again analytic. A formula for the nth derivative of the standard logistic function is \frac{d^n f}{dx^n} = \sum_{i=1}^n \frac{\left(\sum_{j=1}^n {\left(-1\right)}^{i+j} \binom{i}{j} j^n\right) e^{-ix}}{{\left(1+e^{-x}\right)}^{i+1}} therefore its Taylor series about the point a is f(x) = f(a) + \sum_{n=1}^{\infty} \sum_{i=1}^n \frac{\left(\sum_{j=1}^n {\left(-1\right)}^{i+j} \binom{i}{j} j^n\right) e^{-ia}}{{\left(1 + e^{-a}\right)}^{i+1}} \frac{{\left(x-a\right)}^n}{n!} . \frac{d}{dx}f(x) = f(x)\big(1 - f(x)\big) with boundary condition f(0) = 1/2 . This equation is the continuous version of the logistic map. Note that the reciprocal logistic function is solution to a simple first-order linear ordinary differential equation. The qualitative behavior is easily understood in terms of the phase line: the derivative is 0 when the function is 1; and the derivative is positive for f between 0 and 1, and negative for f above 1 or less than 0 (though negative populations do not generally accord with a physical model). This yields an unstable equilibrium at 0 and a stable equilibrium at 1, and thus for any function value greater than 0 and less than 1, it grows to 1. The logistic equation is a special case of the Bernoulli differential equation and has the following solution: f(x) = \frac{e^x}{e^x + C}. Choosing the constant of integration C = 1 gives the other well known form of the definition of the logistic curve: f(x) = \frac{e^x}{e^x + 1} = \frac{1}{1 + e^{-x}}. More quantitatively, as can be seen from the analytical solution, the logistic curve shows early exponential growth for negative argument, which reaches to linear growth of slope 1/4 for an argument near 0, then approaches 1 with an exponentially decaying gap. The differential equation derived above is a special case of a general differential equation that only models the sigmoid function for x > 0 . In many modeling applications, the more general form \frac{df(x)}{dx} = \frac{k}{L} f(x)\big(L - f(x)\big), \quad f(0) = \frac {L} {1 + e^{k x_0}} can be desirable. Its solution is the shifted and scaled sigmoid function L \sigma \big(k(x - x_0)\big) = \frac {L} {1 + e^{-k(x - x_0)}} .

Probabilistic interpretation

When the capacity L = 1 , the value of the logistic function is in the range and can be interpreted as a probability . In more detail, can be interpreted as the probability of one of two alternatives (the parameter of a Bernoulli distribution); the two alternatives are complementary, so the probability of the other alternative is q = 1 - p and p + q = 1 . The two alternatives are coded as 1 and 0, corresponding to the limiting values as x \to \pm \infty . In this interpretation the input is the log-odds for the first alternative (relative to the second, measured in "logistic units" or logits), and so is the odds for the first alternative (relative to the second). Given odds for an event of O = O:1 ( against ), the probability is the ratio of "for" over "for plus against", O/(O+1) . We see that the logistic function, e^x/(e^x + 1) = 1/(1 + e^{-x}) = p , is the probability of the first alternative. Conversely, is the log-odds against the second alternative, is the log-odds for the second alternative, e^{-x} is the odds for the second alternative, and e^{-x}/(e^{-x} + 1) = 1/(1 + e^x) = q = 1 - p is the probability of the second alternative. This can be framed more symmetrically in terms of two inputs, and , which then generalizes naturally to more than two alternatives. Given two real number inputs, and , interpreted as logits, their difference x_1 - x_0 is the log-odds for option 1 (the log-odds against option 0), e^{x_1 - x_0} is the odds, e^{x_1 - x_0}/(e^{x_1 - x_0} + 1) = 1/\left(1 + e^{-(x_1 - x_0)}\right) = e^{x_1}/(e^{x_0} + e^{x_1}) is the probability of option 1, and similarly e^{x_0}/(e^{x_0} + e^{x_1}) is the probability of option 0. This form immediately generalizes to more alternatives as the softmax function, which is a vector-valued function whose -th coordinate is e^{x_i} / \sum_{i=0}^n e^{x_i} . More subtly, the symmetric form emphasizes interpreting the input as x_1 - x_0 and thus relative to some reference point, implicitly to x_0 = 0 . Notably, the softmax function is invariant under adding a constant to all the logits x_i , which corresponds to the difference x_j - x_i being the log-odds for option against option , but the individual logits x_i not being log-odds on their own. Often one of the options is used as a reference ("pivot"), and its value fixed as , so the other logits are interpreted as odds versus this reference. This is generally done with the first alternative, hence the choice of numbering: x_0 = 0 , and then x_i = x_i - x_0 is the log-odds for option against option . Since e^0 = 1 , this yields the +1 term in many expressions for the logistic function and generalizations.

Generalizations

In growth modeling, numerous generalizations exist, including the generalized logistic curve, the Gompertz function, the cumulative distribution function of the shifted Gompertz distribution, and the hyperbolastic function of type I. In statistics, where the logistic function is interpreted as the probability of one of two alternatives, the generalization to three or more alternatives is the softmax function, which is vector-valued, as it gives the probability of each alternative. \frac{dP}{dt}=r P \left(1 - \frac{P}{K}\right), where the constant r defines the growth rate and K is the carrying capacity. In the equation, the early, unimpeded growth rate is modeled by the first term +rP . The value of the rate r represents the proportional increase of the population P in one unit of time. Later, as the population grows, the modulus of the second term (which multiplied out is -r P^2 / K ) becomes almost as large as the first, as some members of the population P interfere with each other by competing for some critical resource, such as food or living space. This antagonistic effect is called the bottleneck, and is modeled by the value of the parameter K . The competition diminishes the combined growth rate, until the value of P ceases to grow (this is called maturity of the population). The solution to the equation (with P_0 being the initial population) is P(t) = \frac{K P_0 e^{rt}}{K + P_0 \left( e^{rt} - 1\right)} = \frac{K}{1+\left(\frac{K-P_0}{P_0}\right)e^{-rt}}, where \lim_{t\to\infty} P(t) = K, where K is the limiting value of P , the highest value that the population can reach given infinite time (or come close to reaching in finite time). The carrying capacity is asymptotically reached independently of the initial value P(0) > 0 , and also in the case that P(0) > K . In ecology, species are sometimes referred to as r -strategist or K -strategist depending upon the selective processes that have shaped their life history strategies. Choosing the variable dimensions so that n measures the population in units of carrying capacity, and \tau measures time in units of 1/r , gives the dimensionless differential equation \frac{dn}{d\tau} = n (1-n). The antiderivative of the ecological form of the logistic function can be computed by the substitution u = K + P_0 \left( e^{rt} - 1\right) , since du = r P_0 e^{rt} dt \int \frac{K P_0 e^{rt}}{K + P_0 \left( e^{rt} - 1\right)}\,dt = \int \frac{K}{r} \frac{1}{u}\,du = \frac{K}{r} \ln u + C = \frac{K}{r} \ln \left(K + P_0 (e^{rt} - 1) \right) + C \frac{dP}{dt} = rP \cdot \left(1 - \frac{P}{K(t)}\right). A particularly important case is that of carrying capacity that varies periodically with period T : K(t + T) = K(t). It can be shown that in such a case, independently from the initial value P(0) > 0 , P(t) will tend to a unique periodic solution P_*(t) , whose period is T . A typical value of T is one year: In such case K(t) may reflect periodical variations of weather conditions. Another interesting generalization is to consider that the carrying capacity K(t) is a function of the population at an earlier time, capturing a delay in the way population modifies its environment. This leads to a logistic delay equation, which has a very rich behavior, with bistability in some parameter range, as well as a monotonic decay to zero, smooth exponential growth, punctuated unlimited growth (i.e., multiple S-shapes), punctuated growth or alternation to a stationary level, oscillatory approach to a stationary level, sustainable oscillations, finite-time singularities as well as finite-time death. p = f(a + bx), where x is the explanatory variable, a and b are model parameters to be fitted, and f is the standard logistic function. Logistic regression and other log-linear models are also commonly used in machine learning. A generalisation of the logistic function to multiple inputs is the softmax activation function, used in multinomial logistic regression. Another application of the logistic function is in the Rasch model, used in item response theory. In particular, the Rasch model forms a basis for maximum likelihood estimation of the locations of objects or persons on a continuum, based on collections of categorical data, for example the abilities of persons on a continuum based on responses that have been categorized as correct and incorrect. g(h) = \frac{1}{1 + e^{-2 \beta h}}, which is a logistic function. These relationships result in simplified implementations of artificial neural networks with artificial neurons. Practitioners caution that sigmoidal functions which are antisymmetric about the origin (e.g. the hyperbolic tangent) lead to faster convergence when training networks with backpropagation. The logistic function is itself the derivative of another proposed activation function, the softplus. X' = r\left(1 - \frac X K \right)X, which is of the type X' = F(X)X, \quad F'(X) \le 0, where F(X) is the proliferation rate of the tumor. If a course of chemotherapy is started with a log-kill effect, the equation may be revised to be X' = r\left(1 - \frac X K \right)X - c(t) X, where c(t) is the therapy-induced death rate. In the idealized case of very long therapy, c(t) can be modeled as a periodic function (of period T ) or (in case of continuous infusion therapy) as a constant function, and one has that \frac 1 T \int_0^T c(t)\, dt > r \to \lim_{t \to +\infty} x(t) = 0, i.e. if the average therapy-induced death rate is greater than the baseline proliferation rate, then there is the eradication of the disease. Of course, this is an oversimplified model of both the growth and the therapy. For example, it does not take into account the evolution of clonal resistance, or the side-effects of the therapy on the patient. These factors can result in the eventual failure of chemotherapy, or its discontinuation. f(t ; \theta_1,\theta_2,\theta_3, \xi) = \frac{\theta_1}{{\left[1 + \xi \exp \left(-\theta_2 \cdot (t - \theta_3) \right) \right]}^{1/\xi}} where \theta_1,\theta_2,\theta_3 are real numbers, and \xi is a positive real number. The flexibility of the curve f is due to the parameter \xi : (i) if \xi = 1 then the curve reduces to the logistic function, and (ii) as \xi approaches zero, the curve converges to the Gompertz function. In epidemiological modeling, \theta_1 , \theta_2 , and \theta_3 represent the final epidemic size, infection rate, and lag phase, respectively. See the right panel for an example infection trajectory when (\theta_1,\theta_2,\theta_3) is set to (10000,0.2,40) . One of the benefits of using a growth function such as the generalized logistic function in epidemiological modeling is its relatively easy application to the multilevel model framework, where information from different geographic regions can be pooled together.

In material science: phase diagrams

See Diffusion bonding. In The Laws of Imitation (1890), Gabriel Tarde describes the rise and spread of new ideas through imitative chains. In particular, Tarde identifies three main stages through which innovations spread: the first one corresponds to the difficult beginnings, during which the idea has to struggle within a hostile environment full of opposing habits and beliefs; the second one corresponds to the properly exponential take-off of the idea, with f(x)=2^x ; finally, the third stage is logarithmic, with f(x)=\log(x) , and corresponds to the time when the impulse of the idea gradually slows down while, simultaneously new opponent ideas appear. The ensuing situation halts or stabilizes the progress of the innovation, which approaches an asymptote. In a sovereign state, the subnational units (constituent states or cities) may use loans to finance their projects. However, this funding source is usually subject to strict legal rules as well as to economy scarcity constraints, especially the resources the banks can lend (due to their equity or Basel limits). These restrictions, which represent a saturation level, along with an exponential rush in an economic competition for money, create a public finance diffusion of credit pleas and the aggregate national response is a sigmoid curve. Historically, when new products are introduced there is an intense amount of research and development which leads to dramatic improvements in quality and reductions in cost. This leads to a period of rapid industry growth. Some of the more famous examples are: railroads, incandescent light bulbs, electrification, cars and air travel. Eventually, dramatic improvement and cost reduction opportunities are exhausted, the product or process are in widespread use with few remaining potential new customers, and markets become saturated. Logistic analysis was used in papers by several researchers at the International Institute of Applied Systems Analysis (IIASA). These papers deal with the diffusion of various innovations, infrastructures and energy source substitutions and the role of work in the economy as well as with the long economic cycle. Long economic cycles were investigated by Robert Ayres (1989). Cesare Marchetti published on long economic cycles and on diffusion of innovations. Arnulf Grübler's book (1990) gives a detailed account of the diffusion of infrastructures including canals, railroads, highways and airlines, showing that their diffusion followed logistic shaped curves. Carlota Perez used a logistic curve to illustrate the long (Kondratiev) business cycle with the following labels: beginning of a technological era as irruption, the ascent as frenzy, the rapid build out as synergy and the completion as maturity.

Sequential analysis

Link created an extension of Wald's theory of sequential analysis to a distribution-free accumulation of random variables until either a positive or negative bound is first equaled or exceeded. Link derives the probability of first equaling or exceeding the positive boundary as 1/(1+e^{-\theta A}) , the logistic function. This is the first proof that the logistic function may have a stochastic process as its basis. Link provides a century of examples of "logistic" experimental results and a newly derived relation between this probability and the time of absorption at the boundaries.

See also

Cross fluid Hyperbolic growth Heaviside step function Hill equation (biochemistry) Hubbert curve List of mathematical functions STAR model Michaelis–Menten kinetics r/K selection theory Rectifier (neural networks) Shifted Gompertz distribution Tipping point (sociology)

Text derived from https://en.wikipedia.org/w/index.php?oldid=1372153319 under CC BY-SA 4.0; modifications have been made.

Sources cited in this revision

24 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.