ramp function

of the ramp function The ramp function is a unary real function, whose graph is shaped like a ramp. It can be expressed by numerous definitions, for example "0 for negative inputs, output equals input for non-negative inputs". The term "ramp" can also be used for other functions obtained by scaling and shifting, and the function in this article is the unit ramp function (slope 1, starting at 0). In mathematics, the ramp function is also known as the positive part. In machine learning, it is commonly known as a ReLU activation function or a rectifier in analogy to half-wave rectification in electrical engineering. In statistics (when used as a likelihood function) it is known as a tobit model. This function has numerous applications in mathematics and engineering, and goes by various names, depending on the context. There are differentiable variants of the ramp function.

of the ramp function The ramp function is a unary real function, whose graph is shaped like a ramp. It can be expressed by numerous definitions, for example "0 for negative inputs, output equals input for non-negative inputs". The term "ramp" can also be used for other functions obtained by scaling and shifting, and the function in this article is the unit ramp function (slope 1, starting at 0). In mathematics, the ramp function is also known as the positive part. In machine learning, it is commonly known as a ReLU activation function or a rectifier in analogy to half-wave rectification in electrical engineering. In statistics (when used as a likelihood function) it is known as a tobit model. This function has numerous applications in mathematics and engineering, and goes by various names, depending on the context. There are differentiable variants of the ramp function. A piecewise function: R(x) := \begin{cases} x, & x \ge 0; \\ 0, & x Using the Iverson bracket notation: R(x) := x \cdot [x \geq 0] or R(x) := x \cdot [x > 0] The max function: R(x) := \max(x,0) The mean of an independent variable and its absolute value (a straight line with unity gradient and its modulus): R(x) := \frac{x+|x|}{2} this can be derived by noting the following definition of , \max(a,b) = \frac{a + b + |a - b|}{2} for which and The Heaviside step function multiplied by a straight line with unity gradient: R\left( x \right) := x H(x) The convolution of the Heaviside step function with itself: R\left( x \right) := H(x) * H(x) The integral of the Heaviside step function: R(x) := \int_{-\infty}^{x} H(\xi)\,d\xi Macaulay brackets: R(x) := \langle x\rangle The positive part of the identity function: R := \operatorname{id}^+ As a limit function: R\left( x \right) := \lim_{a\to \infty} \begin{cases} \frac{1}{a} ,\quad x=0 \\ \dfrac{x}{1-e^{-ax}},\quad x\neq 0\end{cases} It could approximated as close as desired by choosing an increasing positive value a>0 . with a knot at x=3.1 In statistics, hinge functions of multivariate adaptive regression splines (MARS) are ramps, and are used to build regression models. \forall x \in \Reals: R(x) \geq 0 and \left| R (x) \right| = R(x) R'(x) = H(x)\quad \mbox{for } x \ne 0. \frac{d^2}{dx^2} R(x - x_0) = \delta(x - x_0), where is the Dirac delta. This means that is a Green's function for the second derivative operator. Thus, any function, , with an integrable second derivative, , will satisfy the equation: f(x) = f(a) + (x-a) f'(a) + \int_{a}^b R(x - s) f(s) \,ds \quad \mbox{for }a \int {R(x)dx} = \frac{x^2 + x\vert x \vert}{4} + C = \frac{x}{2}R(x) +C\,, where C is the integration constant. \mathcal{F}\big\{ R(x) \big\}(f) = \int_{-\infty}^{\infty} R(x) e^{-2\pi ifx} \, dx = \frac{i\delta '(f)}{4\pi}-\frac{1}{4 \pi^2 f^2}, where is the Dirac delta (in this formula, its derivative appears). \mathcal{L}\big\{R(x)\big\} (s) = \int_{0}^{\infty} e^{-sx}R(x)dx = \frac{1}{s^2}. R \big( R(x) \big) = R(x) . ==See also== Tobit model Rectifier (neural networks)

Text derived from https://en.wikipedia.org/w/index.php?oldid=1352857425 under CC BY-SA 4.0; modifications have been made.