Kurtosis
≈ 17 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
Kurtosis (from ( or ), meaning 'curved, arching') refers to the degree of tailedness in the probability distribution of a real-valued, random variable in probability theory and statistics. Similar to skewness, kurtosis provides insight into specific characteristics of a distribution.
Various methods exist for quantifying kurtosis in theoretical distributions, and corresponding techniques allow estimation based on sample data from a population. Different measures of kurtosis can yield varying interpretations. The standard measure of a distribution's kurtosis, originating with Karl Pearson, is a scaled version of the fourth moment of the distribution.
This number is related to the tails of the distribution, not its peak; hence, the sometimes-seen characterization of kurtosis as peakedness is incorrect. For this measure, higher kurtosis corresponds to greater extremity of deviations (or outliers), and not the configuration of data near the mean.
The widespread misunderstanding of kurtosis may derive from Pearson's own description of kurtosis, as quoted by Westfall, as a departure from normality, a : 'degree of flat-toppedness which is greater or less than that of the normal curve'. Excess kurtosis, typically compared to a value of 0, characterizes the tailedness of a distribution.
A univariate normal distribution has an excess kurtosis of 0.…
Kurtosis (from ( or ), meaning 'curved, arching') refers to the degree of tailedness in the probability distribution of a real-valued, random variable in probability theory and statistics. Similar to skewness, kurtosis provides insight into specific characteristics of a distribution.
Various methods exist for quantifying kurtosis in theoretical distributions, and corresponding techniques allow estimation based on sample data from a population. Different measures of kurtosis can yield varying interpretations. The standard measure of a distribution's kurtosis, originating with Karl Pearson, is a scaled version of the fourth moment of the distribution.
This number is related to the tails of the distribution, not its peak; hence, the sometimes-seen characterization of kurtosis as peakedness is incorrect. For this measure, higher kurtosis corresponds to greater extremity of deviations (or outliers), and not the configuration of data near the mean.
The widespread misunderstanding of kurtosis may derive from Pearson's own description of kurtosis, as quoted by Westfall, as a departure from normality, a : 'degree of flat-toppedness which is greater or less than that of the normal curve'. Excess kurtosis, typically compared to a value of 0, characterizes the tailedness of a distribution.
A univariate normal distribution has an excess kurtosis of 0. Negative excess kurtosis indicates a platykurtic distribution, which does not necessarily have a flat top but produces fewer or less extreme outliers than the normal distribution. For instance, the uniform distribution (i.e., one that is uniformly finite over some bound and zero elsewhere) is platykurtic.
On the other hand, positive excess kurtosis signifies a leptokurtic distribution. The Laplace distribution for example, has tails that decay more slowly than a normal one, resulting in more outliers. To simplify comparison with the normal distribution, excess kurtosis is calculated as Pearson's kurtosis minus 3.
Some authors and software packages use kurtosis to refer specifically to excess kurtosis, but this article distinguishes between the two for clarity. Alternative measures of kurtosis are: the L-kurtosis, which is a scaled version of the fourth L-moment; measures based on four population or sample quantiles.
These are analogous to the alternative measures of skewness that are not based on ordinary moments. \begin{align} \operatorname{Kurt}[X] &:= \tilde{\mu}_4 \equiv \frac{\mu_4}{\sigma^4}\\ &= \operatorname{E}\left[{\left(\frac{X - \mu}{\sigma}\right)}^4\right] = \frac{\operatorname{E}\left[(X - \mu)^4\right]}{\left(\operatorname{E}\left[(X - \mu)^2\right]\right)^2}\\ \end{align} where is the fourth central moment and is the standard deviation. Several letters are used in the literature to denote the kurtosis.
A very common choice is , which is fine as long as it is clear that it does not refer to a cumulant. Other choices include , to be similar to the notation for skewness, although sometimes this is instead reserved for the excess kurtosis. Pearson is systematically using .
The kurtosis is bounded below by the squared skewness plus 1: \frac{\mu_4}{\sigma^4} \geq \left(\frac{\mu_3}{\sigma^3}\right)^2 + 1, where is the third central moment. The lower bound is realized by the Bernoulli distribution. There is no upper limit to the kurtosis of a general probability distribution, and it may be infinite.
A reason why some authors favor the excess kurtosis is that cumulants are extensive. Formulas related to the extensive property are more naturally expressed in terms of the excess kurtosis. For example, let be independent random variables for which the fourth moment exists, and let be the random variable defined by the sum of the .
The excess kurtosis of is \operatorname{Kurt}[Y] - 3 = \frac{\sum_{i=1}^n \sigma_i^{\,4} \cdot \left(\operatorname{Kurt}\left[X_i\right] - 3\right)}{\left( \sum_{j=1}^n \sigma_j^{\,2}\right)^2}, where \sigma_i is the standard deviation of . In particular if all of the have the same variance, then this simplifies to \operatorname{Kurt}[Y] - 3 = \frac{1}{n^2} \sum_{i=1}^n \left(\operatorname{Kurt}\left[X_i\right] - 3\right).
The reason not to subtract 3 is that the bare moment better generalizes to multivariate distributions, especially when independence is not assumed. The cokurtosis between pairs of variables is an order four tensor.
For a bivariate normal distribution, the cokurtosis tensor has off-diagonal terms that are neither 0 nor 3 in general, so attempting to "correct" for an excess becomes confusing. It is true, however, that the joint cumulants of degree greater than two for any multivariate normal distribution are zero.
For two random variables, and , not necessarily independent, the kurtosis of the sum, , is \begin{align} \operatorname{Kurt}[X+Y] &= \frac{1}{\sigma_{X+Y}^4} \big(\sigma_X^4\operatorname{Kurt}[X] \\ & {} + 4\sigma_X^3 \sigma_Y \operatorname{Cokurt}[X,X,X,Y] \\[6pt] & {} + 6\sigma_X^2 \sigma_Y^2 \operatorname{Cokurt}[X,X,Y,Y] \\[6pt] & {} + 4\sigma_X \sigma_Y^3 \operatorname{Cokurt}[X,Y,Y,Y] \\[6pt] & {} + \sigma_Y^4 \operatorname{Kurt}[Y] \big). \end{align} Note that the fourth-power binomial coefficients (1, 4, 6, 4, 1) appear in the above equation.
Moors' interpretation
In 1986, Moors gave an interpretation of kurtosis. Let Z = \frac{ X - \mu } \sigma, where is a random variable, is the mean and is the standard deviation.
Now by definition of the kurtosis \kappa , and by the well-known identity \operatorname{E}\left[V^2\right] = \operatorname{var}[V] + \operatorname{E}[V]^2, \begin{align} \kappa & = \operatorname{E}\left[ Z^4 \right] \\ & = \operatorname{var}\left[ Z^2 \right] + \operatorname{E}{\!\left[Z^2\right]}^2 \\ & = \operatorname{var}\left[ Z^2 \right] + \operatorname{var}[Z]^2 = \operatorname{var}\left[ Z^2 \right] + 1. \end{align} The kurtosis can now be seen as a measure of the dispersion of around its expectation. Alternatively it can be seen to be a measure of the dispersion of around and . attains its minimal value in a symmetric two-point distribution.
In terms of the original variable , the kurtosis is a measure of the dispersion of around the two values . High values of arise where the probability mass is concentrated around the mean and the data-generating process produces occasional values far from the mean, or where the probability mass is concentrated in the tails of the distribution.
The entropy of a distribution is -\!\int p(x) \ln p(x) \, dx. For any \mu \in \R^n, \Sigma \in \R^{n\times n} with \Sigma positive definite, among all probability distributions on \R^n with mean \mu and covariance \Sigma , the normal distribution \mathcal N(\mu, \Sigma) has the largest entropy.
Since mean \mu and covariance \Sigma are the first two moments, it is natural to consider extension to higher moments. In fact, by Lagrange multiplier method, for any prescribed first n moments, if there exists some probability distribution of form p(x) \propto e^{\sum_i a_i x_i + \sum_{ij} b_{ij} x_i x_j + \cdots + \sum_{i_1 \cdots i_n} x_{i_1} \cdots x_{i_n}} that has the prescribed moments (if it is feasible), then it is the maximal entropy distribution under the given constraints.
By serial expansion, \begin{align} & \int \frac{1}{\sqrt{2\pi}} e^{-\frac 12 x^2 - \frac 14 gx^4} x^{2n} \, dx \\[6pt] &= \frac{1}{\sqrt{2\pi}} \int e^{-\frac 12 x^2 - \frac 14 gx^4} x^{2n} \, dx \\[6pt] &= \sum_k \frac{1}{k!} \left(-\frac{g}{4}\right)^k (2n+4k-1)!! \\[6pt] &= (2n-1)!! - \tfrac{1}{4} g (2n+3)!! + O(g^2) \end{align} so if a random variable has probability distribution p(x) = e^{-\frac 12 x^2 - \frac 14 gx^4}/Z , where Z is a normalization constant, then its kurtosis is . Distributions with zero excess kurtosis are called mesokurtic or mesokurtotic.
The most prominent example of a mesokurtic distribution is the normal distribution family, regardless of the values of its parameters. A few other well-known distributions can be mesokurtic, depending on parameter values: for example, the binomial distribution is mesokurtic for p = 1/2 \pm \sqrt{1/12} . A distribution with negative excess kurtosis is called platykurtic, or platykurtotic.
A platykurtic distribution has thinner tails ( means 'broad', originally referring to the peak). Examples of platykurtic distributions include the continuous and discrete uniform distributions, and the raised cosine distribution.
The most platykurtic distribution of all is the Bernoulli distribution with p = 1/2 (for example the number of times one obtains heads when flipping a coin once, a coin toss), for which the excess kurtosis is −2. f(x; a, m) = \frac{\Gamma(m)}{a\,\sqrt{\pi}\,\Gamma(m-1/2)} \left[1+\left(\frac{x}{a}\right)^2 \right]^{-m}, where is a scale parameter and is a shape parameter. All densities in this family are symmetric.
The -th moment exists provided . For the kurtosis to exist, we require . Then the mean and skewness exist and are both identically zero.
Setting makes the variance equal to unity. Then the only free parameter is , which controls the fourth moment (and cumulant) and hence the kurtosis. One can reparameterize with m = 5/2 + 3/\gamma_2 , where \gamma_2 is the excess kurtosis as defined above.
This yields a one-parameter leptokurtic family with zero mean, unit variance, zero skewness, and arbitrary non-negative excess kurtosis. The reparameterized density is g(x; \gamma_2) = f{\left(x;\; a = \sqrt{2 + 6 \gamma_2^{-1}},\; m = \tfrac{5}{2} + 3\gamma_2^{-1} \right)}.
In the limit as \gamma_2 \to \infty , one obtains the density g(x) = 3\left(2 + x^2\right)^{-5/2}, which is shown as the red curve in the images on the right. In the other direction as \gamma_2 \to 0 one obtains the standard normal density as the limiting distribution, shown as the black curve.
In the images on the right, the blue curve represents the density x \mapsto g(x; 2) with excess kurtosis of 2. The top image shows that leptokurtic densities in this family have a higher peak than the mesokurtic normal density, although this conclusion is only valid for this select family of distributions.
The comparatively fatter tails of the leptokurtic densities are illustrated in the second image, which plots the natural logarithm of the Pearson type VII densities: the black curve is the logarithm of the standard normal density, which is a parabola. One can see that the normal density allocates little probability mass to the regions far from the mean (has thin tails), compared with the blue curve of the leptokurtic Pearson type VII density with excess kurtosis of 2.
Between the blue curve and the black are other Pearson type VII densities with = 1, 1/2, 1/4, 1/8, and 1/16. The red curve again shows the upper limit of the Pearson type VII family, with \gamma_2 = \infty (which, strictly speaking, means that the fourth moment does not exist). The red curve decreases the slowest as one moves outward from the origin (has fat tails).
D: Laplace distribution, also known as the double exponential distribution, red curve (two straight lines in the log-scale plot), excess kurtosis = 3 S: hyperbolic secant distribution, orange curve, excess kurtosis = 2 L: logistic distribution, green curve, excess kurtosis = 1.2 N: normal distribution, black curve (inverted parabola in the log-scale plot), excess kurtosis = 0 C: raised cosine distribution, cyan curve, excess kurtosis = −0.593762... W: Wigner semicircle distribution, blue curve, excess kurtosis = −1 U: uniform distribution, magenta curve (shown for clarity as a rectangle in both images), excess kurtosis = −1.2.
Note that in these cases the platykurtic densities have bounded support, whereas the densities with positive or zero excess kurtosis are supported on the whole real line. One cannot infer that high or low kurtosis distributions have the characteristics indicated by these examples.
There exist platykurtic densities with infinite support, e.g., exponential power distributions with sufficiently large shape parameter b, and there exist leptokurtic densities with finite support. An example of the latter is a distribution that is uniform between −3 and −0.3, between −0.3 and 0.3, and between 0.3 and 3, with the same density in the (−3, −0.3) and (0.3, 3) intervals, but with 20 times more density in the (−0.3, 0.3) interval.
Also, one cannot infer from the graphs that higher kurtosis distributions are more peaked and that lower kurtosis distributions are more flat. There exist platykurtic densities with infinite peakedness; e.g., an equal mixture of the beta distribution with parameters 0.5 and 1 with its reflection about 0.0, and there exist leptokurtic densities that appear flat-topped; e.g., a mixture of distribution that is uniform between −1 and 1 with a T(4.0000001) Student's t-distribution, with mixing probabilities 0.999 and 0.001.
Graphs of the standardized versions of these distributions are given to the right. g_2 \equiv \frac{m_4}{m_2^2} -3 = \frac{\tfrac{1}{n} \sum_{i=1}^n \left(x_i - \overline{x}\right)^4}{\left[\tfrac{1}{n} \sum_{i=1}^n \left(x_i - \overline{x}\right)^2\right]^2} - 3 where is the fourth sample moment about the mean, is the second sample moment about the mean (that is, the sample variance), is the -th value, and \overline{x} is the sample mean. This formula has the simpler representation, g_2 = \frac{1}{n} \sum_{i=1}^n z_i^4 - 3 where the z_i values are the standardized data values using the standard deviation defined using rather than in the denominator.
For example, suppose the data values are 0, 3, 4, 1, 2, 3, 0, 2, 1, 3, 2, 0, 2, 2, 3, 2, 5, 2, 3, 999. Then the values are −0.239, −0.225, −0.221, −0.234, −0.230, −0.225, −0.239, −0.230, −0.234, −0.225, −0.230, −0.239, −0.230, −0.230, −0.225, −0.230, −0.216, −0.230, −0.225, 4.359 and the values are 0.003, 0.003, 0.002, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.003, 0.002, 0.003, 0.003, 360.976.
The average of these values is 18.05 and the excess kurtosis is thus . This example makes it clear that data near the middle or peak of the distribution do not contribute to the kurtosis statistic, hence kurtosis does not measure peakedness.
It is simply a measure of the outlier, 999 in this example. \begin{align} &G_2 \equiv \frac{k_4}{k_2^2}\\ &= \frac{n^2\,\left[(n+1)\,m_4 - 3\,(n-1)\,m_2^2\right]}{(n-1)\,(n-2)\,(n-3)} \; \frac{(n-1)^2}{n^2\,m_2^2} \\[6pt] & = \frac{n-1}{(n-2)\,(n-3)} \left[(n+1)\,\frac{m_4}{m_2^2} - 3\,(n-1) \right] \\[6pt] & = \frac{n-1}{(n-2)\,(n-3)} \left[(n+1)\,g_2 + 6 \right] \end{align} where is the unique symmetric unbiased estimator of the fourth cumulant, is the unbiased estimate of the second cumulant (identical to the unbiased estimate of the sample variance), is the fourth sample moment about the mean, is the second sample moment about the mean, is the -th value, and \bar{x} is the sample mean. This adjusted Fisher–Pearson standardized moment coefficient G_2 is the version found in Excel and several statistical packages including Minitab, SAS, and SPSS.
Unfortunately, in non-normal samples G_2 is itself generally biased. g_2 \le \frac{1}{2} \frac{n-3}{n-2} g_1^2 + \frac{n}{2} - 3, where g_1 = m_3/m_2^{3/2} is the corresponding sample skewness. The variance of the sample kurtosis of a sample of size from the normal distribution is \operatorname{var}(g_2) = \frac{24n(n-1)^2}{(n-3)(n-2)(n+3)(n+5)} Stated differently, under the assumption that the underlying random variable X is normally distributed, it can be shown that \sqrt{n} g_2 \,\xrightarrow{d}\, \mathcal{N}(0, 24) .
Assume a random variable has expectation \operatorname{E}[X] = \mu , variance \operatorname{E}\left[(X - \mu)^2\right] = \sigma^2 and kurtosis \kappa = \tfrac{1}{\sigma^4} \operatorname{E}\left[(X - \mu)^4\right]. Assume we sample n = \tfrac{2\sqrt{3} + 3}{3} \kappa \log\tfrac{1}{\delta} many independent copies.
Then \begin{align} &\Pr\left[\max_{i=1}^n X_i \le \mu\right] \le \delta\\[4pt] &\text{and}\\[4pt] & \Pr\left[\min_{i=1}^n X_i \ge \mu\right] \le \delta. \end{align} This shows that with \Theta(\kappa\log\tfrac{1}\delta) many samples, we will see one that is above the expectation with probability at least 1-\delta . In other words: If the kurtosis is large, there may be a lot of values either all below or above the mean.
Kurtosis convergence
Applying band-pass filters to digital images, kurtosis values tend to be uniform, independent of the range of the filter. This behavior, termed kurtosis convergence, can be used to detect image splicing in forensic analysis. Kurtosis calculator Kurtosis on the Earliest known uses of some of the words of mathematics Celebrating 100 years of Kurtosis a history of the topic, with different measures of kurtosis.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1376046239 under CC BY-SA 4.0; modifications have been made.