Lesson plan
Rectified linear unit
(green) functions near x = 0 In the context of artificial neural networks, the rectifier or ReLU (rectified linear unit) activation function is an activation function defined as the non-negative part of its argument, i.e., the ramp function: : \operatorname{ReLU}(x) = x^+ = \max(0, x) = \frac{x+|x|}{2} = \begin{cases} x & \text{if } x > 0, \\ 0 & x \le 0 \end{cases} where x is the input to a neuron. This is analogous to half-wave rectification in electrical engineering. ReLU is one of the most popular activation functions for artificial neural networks, and finds application in computer vision and speech recognition using deep neural nets and computational neuroscience.
Article summary
(green) functions near x = 0 In the context of artificial neural networks, the rectifier or ReLU (rectified linear unit) activation function is an activation function defined as the non-negative part of its argument, i.e., the ramp function: : \operatorname{ReLU}(x) = x^+ = \max(0, x) = \frac{x+|x|}{2} = \begin{cases} x & \text{if } x > 0, \\ 0 & x \le 0 \end{cases} where x is the input to a neuron. This is analogous to half-wave rectification in electrical engineering. ReLU is one of the most popular activation functions for artificial neural networks, and finds application in computer vision and speech recognition using deep neural nets and computational neuroscience.
Key claims
No claims recorded.
Discussion questions
- Summarise the article in your own words. Which single claim would you verify first, and why?
- Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
- If two sources disagreed about one of these facts, what would you need in order to trust one over the other?
Bibliography
No sources recorded.
Key claims and bibliography are generated from the same tracked claims and sources as the article page.