Characteristic function (probability theory)
≈ 15 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In probability theory and statistics, the characteristic function of any real-valued random variable completely defines its probability distribution. If a random variable admits a probability density function, then the characteristic function is the Fourier transform (with sign reversal) of the probability density function. Thus it provides an alternative route to analytical results compared with working directly with probability density functions or cumulative distribution functions. There are particularly simple results for the characteristic functions of distributions defined by the weighted sums of random variables. In addition to univariate distributions, characteristic functions can be defined for multivariate probability distributions of vector- or matrix-valued random variables, and can also be extended to more generic cases. The characteristic function always exists when treated as a function of a real-valued argument, unlike the moment-generating function. There are relations between the behavior of the characteristic function of a distribution and properties of the distribution, such as the existence of moments and the existence of a density function.
In probability theory and statistics, the characteristic function of any real-valued random variable completely defines its probability distribution. If a random variable admits a probability density function, then the characteristic function is the Fourier transform (with sign reversal) of the probability density function. Thus it provides an alternative route to analytical results compared with working directly with probability density functions or cumulative distribution functions. There are particularly simple results for the characteristic functions of distributions defined by the weighted sums of random variables. In addition to univariate distributions, characteristic functions can be defined for multivariate probability distributions of vector- or matrix-valued random variables, and can also be extended to more generic cases. The characteristic function always exists when treated as a function of a real-valued argument, unlike the moment-generating function. There are relations between the behavior of the characteristic function of a distribution and properties of the distribution, such as the existence of moments and the existence of a density function. \varphi_X(t) = \operatorname{E} \left [ e^{itX} \right ], a function of , determines the behavior and properties of the probability distribution of . It is equivalent to a probability density function (if it exists) or cumulative distribution function, in the sense that knowing one of these functions allows computation of the others, but they provide different insights into the features of the random variable. In particular cases, one or another of these equivalent functions may be easier to represent in terms of simple standard functions. If a random variable admits a density function, then the characteristic function is its Fourier dual, in the sense that each of them is a Fourier transform of the other. If a random variable has a moment-generating function M_X(t) , then the domain of the characteristic function can be extended to the complex plane, and \varphi_X(-it) = M_X(t). Note however that the characteristic function of a distribution is well defined for all real values of , even when the moment-generating function is not well defined for all real values of . The characteristic function approach is particularly useful in analysis of linear combinations of independent random variables: a classical proof of the Central Limit Theorem uses characteristic functions and Lévy's continuity theorem. Another important application is to the theory of the decomposability of random variables. Let X be a real-valued random variable. Its characteristic function \varphi_X:\mathbb{R}\to\mathbb{C} is defined for each t\in\mathbb{R} as the expected value of e^{itX} , where i is the imaginary unit. \varphi_X(t) = \operatorname{E}\left[e^{itX}\right] = \int_{\mathbb{R}} e^{itx}\,dF_X(x) = \int_0^1 e^{it Q_X(p)}\,dp = \int_{\mathbb{R}} e^{itx} f_X(x)\,dx. In the equalites, there is the cumulative distribution function F_X , the inverse cumulative distribution function Q_X (also called the quantile function ), and the density function f_X . The integrals are in the Riemann–Stieltjes sense. The last equality holds if f_X exists, which then implies that \varphi_X is its Fourier transform with sign reversal in the complex exponential. The convention for the constant appearing in the above definition differs from the usual convention for the Fourier transform. For example, some authors define \varphi_X=\left[e^{-2\pi itX}\right] , which is essentially a change of parameter. Other notation may be encountered in the literature: \hat{p} as the characteristic function for a probability measure p , or \hat{f} as the characteristic function corresponding to a density f .
Generalizations
The notion of characteristic functions generalizes to multivariate random variables and more complicated random elements. The argument of the characteristic function will always belong to the continuous dual of the space where the random variable takes its values. For common cases such definitions are listed below: If is a -dimensional random vector, then for \varphi_X(t) = \operatorname{E}\left[\exp( i t^T\!X)\right], where t^T is the transpose of the vector t , If is a -dimensional random matrix, then for \varphi_X(t) = \operatorname{E}\left[\exp \left( i \operatorname{tr}(t^T\!X) \right )\right], where \operatorname{tr}(\cdot) is the trace operator, If is a complex random variable, then for \varphi_X(t) = \operatorname{E}\left[\exp\left( i \operatorname{Re}\left(\overline{t}X\right) \right)\right], where \overline t is the complex conjugate of t and \operatorname{Re}(z) is the real part of the complex number z , If is a -dimensional complex random vector, then for \varphi_X(t) = \operatorname{E}\left[\exp(i\operatorname{Re}(t^*\!X))\right], where t^* is the conjugate transpose of the vector t , If is a stochastic process, then for all functions such that the integral \int_{\mathbb R} t(s)X(s)\,\mathrm{d}s converges for almost all realizations of \varphi_X(t) = \operatorname{E}\left[\exp \left ( i\int_\mathbf{R} t(s)X(s) \, ds \right ) \right]. There is a bijection between probability distributions and characteristic functions on \mathbf{R}^k , k\in\N . That is, for any two random variables , , with values in \mathbf{R}^k , both have the same probability distribution if and only if \varphi_{X_1}=\varphi_{X_2} . If a random variable has moments up to -th order, then the characteristic function is times continuously differentiable on the entire real line. In this case \operatorname{E}[X^k] = i^{-k} \varphi_X^{(k)}(0). If a characteristic function has a -th derivative at zero, then the random variable has all moments up to if is even, but only up to if is odd. \varphi_X^{(k)}(0) = i^k \operatorname{E}[X^k] If are independent random variables, and are some constants, then the characteristic function of the linear combination of the variables is \varphi_{a_1X_1+\cdots+a_nX_n}(t) = \varphi_{X_1}(a_1t)\cdots \varphi_{X_n}(a_nt). One specific case is the sum of two independent random variables and in which case one has \varphi_{X_1+X_2}(t) = \varphi_{X_1}(t)\cdot\varphi_{X_2}(t). Let X and Y be two random variables with characteristic functions \varphi_{X} and \varphi_{Y} . X and Y are independent if and only if \varphi_{X, Y}(s, t)= \varphi_{X}(s) \varphi_{Y}(t) \quad \text { for all } \quad(s, t) \in \mathbb{R}^{2} . The tail behavior of the characteristic function determines the smoothness of the corresponding density function. Let the random variable Y = aX + b be the linear transformation of a random variable X . The characteristic function of Y is \varphi_Y(t)=e^{itb}\varphi_X(at) . For random vectors X and Y = AX + B (where is a constant matrix and a constant vector), we have \varphi_Y(t) = e^{it^\top B}\varphi_X(A^\top t) . f_X(x) = F_X'(x) = \frac{1}{2\pi}\int_{\mathbf{R}} e^{-itx}\varphi_X(t)\,dt. In the multivariate case it is f_X(x) = \frac{1}{(2\pi)^n} \int_{\mathbf{R}^n} e^{-i(t\cdot x)}\varphi_X(t)\lambda(dt) where t\cdot x is the dot product. The density function is the Radon–Nikodym derivative of the distribution with respect to the Lebesgue measure : f_X(x) = \frac{d\mu_X}{d\lambda}(x). Theorem (Lévy). If is characteristic function of distribution function , two points are such that is a continuity set of (in the univariate case this condition is equivalent to continuity of at points and ), then If is scalar: F_X(b) - F_X(a) = \frac{1} {2\pi} \lim_{T \to \infty} \int_{-T}^{+T} \frac{e^{-ita} - e^{-itb}} {it}\, \varphi_X(t)\, dt. This formula can be re-stated in a form more convenient for numerical computation as \frac{F(x+h) - F(x-h)}{2h} = \frac{1}{2\pi} \int_{-\infty}^{\infty} \frac{\sin ht}{ht} e^{-itx} \varphi_X(t) \, dt . For a random variable bounded from below one can obtain F(b) by taking a such that F(a)=0. Otherwise, if a random variable is not bounded from below, the limit for a\to-\infty gives F(b) , but is numerically impractical. If is a vector random variable: \mu_X\big(\{a Theorem. If is (possibly) an atom of (in the univariate case this means a point of discontinuity of ) then If is scalar: F_X(a) - F_X(a-0) = \lim_{T\to\infty}\frac{1}{2T} \int_{-T}^{+T} e^{-ita}\varphi_X(t)\,dt If is a vector random variable: \mu_X(\{a\}) = \lim_{T_1\to\infty}\cdots\lim_{T_n\to\infty} \left(\prod_{k=1}^n\frac{1}{2T_k}\right) \int\limits_{[-T_1, T_1] \times \dots \times [-T_n, T_n]} e^{-i(t\cdot a)}\varphi_X(t)\lambda(dt) Theorem (Gil-Pelaez). For a univariate random variable , if is a continuity point of then F_X(x) = \frac{1}{2} - \frac{1}{\pi}\int_0^\infty \frac{\operatorname{Im}[e^{-itx}\varphi_X(t)]}{t}\,dt where the imaginary part of a complex number z is given by \mathrm{Im}(z) = (z - z^*)/2i . And its density function is: f_X(x) = \frac{1}{\pi}\int_0^\infty \operatorname{Re}[e^{-itx}\varphi_X(t)]\,dt The integral may be not Lebesgue-integrable; for example, when is the discrete random variable that is always 0, it becomes the Dirichlet integral. Inversion formulas for multivariate distributions are available. A convex linear combination \sum_n a_n\varphi_n(t) (with a_n\geq0,\ \sum_n a_n=1 ) of a finite or a countable number of characteristic functions is also a characteristic function. The product of a finite number of characteristic functions is also a characteristic function. The same holds for an infinite product provided that it converges to a function continuous at the origin. If is a characteristic function and is a real number, then \bar{\varphi} , , and are also characteristic functions. It is well known that any non-decreasing càdlàg function with limits , corresponds to a cumulative distribution function of some random variable. There is also interest in finding similar simple criteria for when a given function could be the characteristic function of some random variable. The central result here is Bochner’s theorem, although its usefulness is limited because the main condition of the theorem, non-negative definiteness, is very hard to verify. Other theorems also exist, such as Khinchine’s, Mathias’s, or Cramér’s, although their application is just as difficult. Pólya’s theorem, on the other hand, provides a very simple convexity condition which is sufficient but not necessary. Characteristic functions which satisfy this condition are called Pólya-type. Bochner’s theorem. An arbitrary function is the characteristic function of some random variable if and only if is positive definite, continuous at the origin, and if . Khinchine’s criterion. A complex-valued, absolutely continuous function , with , is a characteristic function if and only if it admits the representation \varphi(t) = \int_{\mathbf{R}} g(t+\theta)\overline{g(\theta)} \, d\theta . Mathias’ theorem. A real-valued, even, continuous, absolutely integrable function , with , is a characteristic function if and only if (-1)^n \left ( \int_{\mathbf{R}} \varphi(pt)e^{-t^2/2} H_{2n}(t) \, dt \right ) \geq 0 for , and all . Here denotes the Hermite polynomial of degree . Pólya’s theorem. If \varphi is a real-valued, even, continuous function which satisfies the conditions \varphi(0) = 1 , \varphi is convex for t>0 , \varphi(\infty) = 0 , then is the characteristic function of an absolutely continuous distribution symmetric about 0. S_n = \sum_{i=1}^n a_i X_i,\,\! where the are constants, then the characteristic function for is given by \varphi_{S_n}(t)=\varphi_{X_1}(a_1t)\varphi_{X_2}(a_2t)\cdots \varphi_{X_n}(a_nt) \,\! In particular, . To see this, write out the definition of characteristic function: \varphi_{X+Y}(t)= \operatorname{E}\left [e^{it(X+Y)}\right]= \operatorname{E}\left [e^{itX}e^{itY}\right] = \operatorname{E}\left [e^{itX}\right] \operatorname{E}\left [e^{itY}\right] =\varphi_X(t) \varphi_Y(t) The independence of and is required to establish the equality of the third and fourth expressions. Another special case of interest for identically distributed random variables is when and then S n is the sample mean. In this case, writing for the mean, \varphi_{\overline{X}}(t)= \varphi_X\!\left(\tfrac{t}{n} \right)^n \operatorname{E}\left[ X^n\right] = i^{-n}\left[\frac{d^n}{dt^n}\varphi_X(t)\right]_{t=0} = i^{-n}\varphi_X^{(n)}(0) ,\! This can be formally written using the derivatives of the Dirac delta function: f_X(x) = \sum_{n=0}^\infty \frac{(-1)^n}{n!}\delta^{(n)}(x)\operatorname{E}[X^n] which allows a formal solution to the moment problem. For example, suppose has a standard Cauchy distribution. Then . This is not differentiable at , showing that the Cauchy distribution has no expectation. Also, the characteristic function of the sample mean of independent observations has characteristic function , using the result from the previous section. This is the characteristic function of the standard Cauchy distribution: thus, the sample mean has the same distribution as the population itself. As a further example, suppose follows a Gaussian distribution i.e. X \sim \mathcal{N}(\mu,\sigma^2) . Then \varphi_{X}(t) = e^{\mu i t - \frac{1}{2} \sigma^2 t^2} and \operatorname{E}\left[ X\right] = i^{-1} \left[\frac{d}{dt}\varphi_X(t)\right]_{t=0} = i^{-1} \left[(i \mu - \sigma^2 t) \varphi_X(t) \right]_{t=0} = \mu A similar calculation shows \operatorname{E}\left[ X^2\right] = \mu^2 + \sigma^2 and is easier to carry out than applying the definition of expectation and using integration by parts to evaluate \operatorname{E}\left[ X^2\right] . The logarithm of a characteristic function is a cumulant generating function, which is useful for finding cumulants; some instead define the cumulant generating function as the logarithm of the moment-generating function, and call the logarithm of the characteristic function the second cumulant generating function. \varphi_X(t)=(1 - \theta i t)^{-k_1},\,\qquad \varphi_Y(t)=(1 - \theta it)^{-k_2} which by independence and the basic properties of characteristic function leads to \varphi_{X+Y}(t)=\varphi_X(t)\varphi_Y(t)=(1 - \theta i t)^{-k_1}(1 - \theta i t)^{-k_2}=\left(1 - \theta i t\right)^{-(k_1+k_2)}. This is the characteristic function of the gamma distribution scale parameter and shape parameter , and we therefore conclude X+Y \sim \Gamma(k_1+k_2,\theta) The result can be expanded to independent gamma distributed random variables with the same scale parameter and we get \forall i \in \{1,\ldots, n\} : X_i \sim \Gamma(k_i,\theta) \qquad \Rightarrow \qquad \sum_{i=1}^n X_i \sim \Gamma\left(\sum_{i=1}^nk_i,\theta\right). \mathbf{E}[(\xi, y)]=\widehat{\mu}(y) = \int_X (x, y)\, d\mu(x), \quad y \in Y. \varphi_X(t) = \langle e^{itX} \rangle = \int_{\mathbf{R}} e^{itx}p(x)\, dx = \overline{\left( \int_{\mathbf{R}} e^{-itx}p(x)\, dx \right)} = \overline{P(t)}, where denotes the continuous Fourier transform of the probability density function . Likewise, may be recovered from through the inverse Fourier transform: p(x) = \frac{1}{2\pi} \int_{\mathbf{R}} e^{itx} P(t)\, dt = \frac{1}{2\pi} \int_{\mathbf{R}} e^{itx} \overline{\varphi_X(t)}\, dt. Indeed, even when the random variable does not have a density, the characteristic function may be seen as the Fourier transform of the measure corresponding to the random variable. Another related concept is the representation of probability distributions as elements of a reproducing kernel Hilbert space via the kernel embedding of distributions. This framework may be viewed as a generalization of the characteristic function under specific choices of the kernel function.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1376247591 under CC BY-SA 4.0; modifications have been made.