Geometric distribution
≈ 17 min read
Names in other languages
- EnglishGeometric Distribution
- nbGeometrisk fordeling
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
{{Infobox probability distribution 2 | name = Geometric | type = mass | pdf_image = | cdf_image = | notation = \mathrm{Geom}(p) | parameters = 0 success probability (real) | support = k trials where k \in \mathbb{N} = \{1, 2, 3, \dotsc\} | pdf = (1 - p)^{k-1}p | cdf = 1-(1 - p)^{\lfloor x\rfloor} for x\geq 1 , 0 for x | mean = \frac{1}{p} | median = \left\lceil \frac{-1}{\log_2(1-p)} \right\rceil (not unique if -1/\log_2(1-p) is an integer) | mode = 1 | variance = \frac{1-p}{p^2} | skewness = \frac{2-p}{\sqrt{1-p}} | kurtosis = 6+\frac{p^2}{1-p} | entropy = \tfrac{-(1-p)\log (1-p) - p \log p}{p} | fisher = \tfrac{1}{p^2 (1-p)} | mgf = \frac{pe^t}{1-(1-p) e^t}, for t | char = \frac{pe^{it}}{1-(1-p)e^{it}} | pgf = \frac{pz}{1-(1-p)z} | parameters2 = 0 success probability (real) | support2 = k failures where k \in \mathbb{N}_0 = \{0, 1, 2, \dotsc\} | pdf2 = (1 - p)^k p | cdf2 = 1-(1 - p)^{\lfloor x\rfloor+1} for x\geq 0 , 0 for x | mean2 = \frac{1-p}{p} | median2 = \left\lceil \frac{-1}{\log_2(1-p)} \right\rceil - 1 (not unique if -1/\log_2(1-p) is an integer) | mode2 = 0 | variance2 = \frac{1-p}{p^2} | skewness2 = \frac{2-p}{\sqrt{1-p}} | kurtosis2 = 6+\frac{p^2}{1-p} | entropy2 = \tfrac{-(1-p)\log (1-p) - p \log p}{p} | fisher2 = \tfrac{1}{p^2 (1-p)} | mgf2 = \frac{p}{1-(1-p)e^t}, for t | char2 = \frac{p}{1-(1-p)e^{it}} | pgf2 = \frac{p}{1-(1-p)z} }} In probability theory and st…
{{Infobox probability distribution 2 | name = Geometric | type = mass | pdf_image = | cdf_image = | notation = \mathrm{Geom}(p) | parameters = 0 success probability (real) | support = k trials where k \in \mathbb{N} = \{1, 2, 3, \dotsc\} | pdf = (1 - p)^{k-1}p | cdf = 1-(1 - p)^{\lfloor x\rfloor} for x\geq 1 , 0 for x | mean = \frac{1}{p} | median = \left\lceil \frac{-1}{\log_2(1-p)} \right\rceil (not unique if -1/\log_2(1-p) is an integer) | mode = 1 | variance = \frac{1-p}{p^2} | skewness = \frac{2-p}{\sqrt{1-p}} | kurtosis = 6+\frac{p^2}{1-p} | entropy = \tfrac{-(1-p)\log (1-p) - p \log p}{p} | fisher = \tfrac{1}{p^2 (1-p)} | mgf = \frac{pe^t}{1-(1-p) e^t}, for t | char = \frac{pe^{it}}{1-(1-p)e^{it}} | pgf = \frac{pz}{1-(1-p)z} | parameters2 = 0 success probability (real) | support2 = k failures where k \in \mathbb{N}_0 = \{0, 1, 2, \dotsc\} | pdf2 = (1 - p)^k p | cdf2 = 1-(1 - p)^{\lfloor x\rfloor+1} for x\geq 0 , 0 for x | mean2 = \frac{1-p}{p} | median2 = \left\lceil \frac{-1}{\log_2(1-p)} \right\rceil - 1 (not unique if -1/\log_2(1-p) is an integer) | mode2 = 0 | variance2 = \frac{1-p}{p^2} | skewness2 = \frac{2-p}{\sqrt{1-p}} | kurtosis2 = 6+\frac{p^2}{1-p} | entropy2 = \tfrac{-(1-p)\log (1-p) - p \log p}{p} | fisher2 = \tfrac{1}{p^2 (1-p)} | mgf2 = \frac{p}{1-(1-p)e^t}, for t | char2 = \frac{p}{1-(1-p)e^{it}} | pgf2 = \frac{p}{1-(1-p)z} }} In probability theory and statistics, the geometric distribution is either one of two discrete probability distributions: The probability distribution of the number X of Bernoulli trials needed to get one success, supported on \mathbb{N} = \{1,2,3,\ldots\} ; The probability distribution of the number Y=X-1 of failures before the first success, supported on \mathbb{N}_0 = \{0, 1, 2, \ldots \} .
These two different geometric distributions should not be confused with each other. Often, the name shifted geometric distribution is adopted for the former one (distribution of X ); however, to avoid ambiguity, it is considered wise to indicate which is intended, by mentioning the support explicitly. The geometric distribution gives the probability that the first occurrence of success requires k independent trials, each with success probability p .
If the probability of success on each trial is p , then the probability that the k -th trial is the first success is \Pr(X = k) = (1-p)^{k-1}p for k=1,2,3,4,\dots The above form of the geometric distribution is used for modeling the number of trials up to and including the first success.
By contrast, the following form of the geometric distribution is used for modeling the number of failures until the first success: \Pr(Y=k) = \Pr(X=k+1) = (1 - p)^k p for k=0,1,2,3,\dots The geometric distribution gets its name because its probabilities follow a geometric sequence. It is sometimes called the Furry distribution after Wendell H. Furry.
Definition
The geometric distribution is the discrete probability distribution that describes when the first success in an infinite sequence of independent and identically distributed Bernoulli trials occurs. Its probability mass function depends on its parameterization and support. When supported on \mathbb{N} , the probability mass function is P(X = k) = (1 - p)^{k-1} p where k = 1, 2, 3, \dotsc is the number of trials and p is the probability of success in each trial. The support may also be \mathbb{N}_0 , defining Y=X-1 .
This alters the probability mass function into P(Y = k) = (1 - p)^k p where k = 0, 1, 2, \dotsc is the number of failures before the first success. An alternative parameterization of the distribution gives the probability mass function P(Y = k) = \left(\frac{P}{Q}\right)^k \left(1-\frac{P}{Q}\right) where P = \frac{1-p}{p} and Q = \frac{1}{p} . An example of a geometric distribution arises from rolling a six-sided die until a "1" appears. Each roll is independent with a 1/6 chance of success.
The number of rolls needed follows a geometric distribution with p=1/6 . \Pr(X>m+n\mid X>n)=\Pr(X>m), and \Pr(Y>m+n\mid Y\geq n)=\Pr(Y>m), where m and n are natural numbers, X is a geometrically distributed random variable defined over \mathbb{N} , and Y is a geometrically distributed random variable defined over \mathbb{N}_0 . Note that these definitions are not equivalent for discrete random variables; Y does not satisfy the first equation and X does not satisfy the second.
Moments and cumulants
The expected value and variance of a geometrically distributed random variable X defined over \mathbb{N} is \operatorname{E}(X) = \frac{1}{p}, \qquad \operatorname{var}(X) = \frac{1-p}{p^2}. With a geometrically distributed random variable Y defined over \mathbb{N}_0 , the expected value changes into \operatorname{E}(Y) = \frac{1-p} p, while the variance stays the same.
For example, when rolling a six-sided die until landing on a "1", the average number of rolls needed is \frac{1}{1/6} = 6 and the average number of failures is The moment generating function of the geometric distribution when defined over \mathbb{N} and \mathbb{N}_0 respectively is \begin{align} M_X(t) &= \frac{pe^t}{1-(1-p)e^t} \\ M_Y(t) &= \frac{p}{1-(1-p)e^t}, t The moments for the number of failures before the first success are given by \begin{align} \mathrm{E}(Y^n) & {} =\sum_{k=0}^\infty (1-p)^k p\cdot k^n \\ & {} =p \operatorname{Li}_{-n}(1-p) & (\text{for }n \neq 0) \end{align} where \operatorname{Li}_{-n}(1-p) is the polylogarithm function.
The cumulant generating function of the geometric distribution defined over \mathbb{N}_0 is K(t) = \ln p - \ln (1 - (1-p)e^t) The cumulants \kappa_r satisfy the recursion \kappa_{r+1} = q \frac{\delta\kappa_r}{\delta q}, r=1,2,\dotsc where q = 1-p , when defined over \operatorname \mathrm{E}(X) = p + (1-p)(1 + \mathrm{E}[X]) , which, when solved for \mathrm{E}(X) , gives: \operatorname E(X) = \frac{1}{p}.
The expected number of failures Y can be found from the linearity of expectation, \mathrm{E}(Y) = \mathrm{E}(X-1) = \mathrm{E}(X) - 1 = \frac 1 p - 1 = \frac{1-p}{p} .
It can also be shown in the following way: \begin{align} \operatorname E(Y) & =p\sum_{k=0}^\infty(1-p)^k k \\ & = p (1-p) \sum_{k=0}^\infty (1-p)^{k-1} k\\ & = p (1-p) \left(-\sum_{k=0}^\infty \frac{d}{dp}\left[(1-p)^k\right]\right) \\ & = p (1-p) \left[\frac{d}{dp}\left(-\sum_{k=0}^\infty (1-p)^k\right)\right] \\ & = p(1-p)\frac{d}{dp}\left(-\frac{1}{p}\right) \\ & = \frac{1-p}{p}. \end{align} The interchange of summation and differentiation is justified by the fact that convergent power series converge uniformly on compact subsets of the set of points where they converge.
Entropy and Fisher's information
Entropy (geometric distribution, failures before success)
Entropy is a measure of uncertainty in a probability distribution. For the geometric distribution that models the number of failures before the first success, the probability mass function is: P(X = k) = (1 - p)^k p, \quad k = 0, 1, 2, \dots The entropy H(X) for this distribution is defined as: \begin{align} H(X) &= - \sum_{k=0}^\infty P(X = k) \ln P(X = k) \\ &= - \sum_{k=0}^\infty (1 - p)^k p \ln \left( (1 - p)^k p \right) \\ &= - \sum_{k=0}^\infty (1 - p)^k p \left[ k \ln(1 - p) + \ln p \right] \\ &= -\log p - \frac{1 - p}{p} \log(1 - p) \end{align} The entropy increases as the probability p decreases, reflecting greater uncertainty as success becomes rarer.
Fisher's information (geometric distribution, failures before success)
Fisher information measures the amount of information that an observable random variable X carries about an unknown parameter p .
For the geometric distribution (failures before the first success), the Fisher information with respect to p is given by: I(p) = \frac{1}{p^2(1 - p)} Proof: The likelihood function for a geometric random variable X is: L(p; X) = (1 - p)^X p The log-likelihood function is: \ln L(p; X) = X \ln(1 - p) + \ln p The score function (first derivative of the log-likelihood w.r.t. p ) is: \frac{\partial}{\partial p} \ln L(p; X) = \frac{1}{p} - \frac{X}{1 - p} The second derivative of the log-likelihood function is: \frac{\partial^2}{\partial p^2} \ln L(p; X) = -\frac{1}{p^2} - \frac{X}{(1 - p)^2} Fisher information is calculated as the negative expected value of the second derivative: \begin{align} I(p) &= -E\left[\frac{\partial^2}{\partial p^2} \ln L(p; X)\right] \\ &= - \left(-\frac{1}{p^2} - \frac{1 - p}{p (1 - p)^2} \right) \\ &= \frac{1}{p^2(1 - p)} \end{align} Fisher information increases as p decreases, indicating that rarer successes provide more information about the parameter p .
Entropy (geometric distribution, trials until success)
For the geometric distribution modeling the number of trials until the first success, the probability mass function is: P(X = k) = (1 - p)^{k - 1} p, \quad k = 1, 2, 3, \dots The entropy H(X) for this distribution is the same as that of version modeling trials until failure, \begin{align} H(X) &= - \log p - \frac{1 - p}{p} \log(1 - p) \end{align}
Fisher's information (geometric distribution, trials until success)
Fisher information for the geometric distribution modeling the number of trials until the first success is given by: I(p) = \frac{1}{p^2(1 - p)} Proof: The likelihood function for a geometric random variable X is: L(p; X) = (1 - p)^{X - 1} p The log-likelihood function is: \ln L(p; X) = (X - 1) \ln(1 - p) + \ln p The score function (first derivative of the log-likelihood w.r.t. p ) is: \frac{\partial}{\partial p} \ln L(p; X) = \frac{1}{p} - \frac{X - 1}{1 - p} The second derivative of the log-likelihood function is: \frac{\partial^2}{\partial p^2} \ln L(p; X) = -\frac{1}{p^2} - \frac{X - 1}{(1 - p)^2} Fisher information is calculated as the negative expected value of the second derivative: \begin{align} I(p) &= -E\left[\frac{\partial^2}{\partial p^2} \ln L(p; X)\right] \\ &= - \left(-\frac{1}{p^2} - \frac{1 - p}{p (1 - p)^2} \right) \\ &= \frac{1}{p^2(1 - p)} \end{align} The probability generating functions of geometric random variables X and Y defined over \mathbb{N} and \mathbb{N}_0 are, respectively, \begin{align} G_X(s) & = \frac{s\,p}{1-s\,(1-p)}, \\[10pt] G_Y(s) & = \frac{p}{1-s\,(1-p)}, \quad |s| The characteristic function \varphi(t) is equal to G(e^{it}) so the geometric distribution's characteristic function, when defined over \mathbb{N} and \mathbb{N}_0 respectively, is \begin{align} \varphi_X(t) &= \frac{pe^{it}}{1-(1-p)e^{it}},\\[10pt] \varphi_Y(t) &= \frac{p}{1-(1-p)e^{it}}. \end{align} The entropy of a geometric distribution with parameter p is -\frac{p \log_2 p + (1-p) \log_2 (1-p)}{p} Given a mean, the geometric distribution is the maximum entropy probability distribution of all discrete probability distributions.
The corresponding continuous distribution is the exponential distribution. The geometric distribution defined on \mathbb{N}_0 is infinitely divisible, that is, for any positive integer n , there exist n independent identically distributed random variables whose sum is also geometrically distributed. This is because the negative binomial distribution can be derived from a Poisson-stopped sum of logarithmic random variables.
The decimal digits of the geometrically distributed random variable Y are a sequence of independent (and not identically distributed) random variables. For example, the hundreds digit D has this probability distribution: \Pr(D=d) = {q^{100d} \over 1 + q^{100} + q^{200} + \cdots + q^{900}}, where , and similarly for the other digits, and, more generally, similarly for numeral systems with other bases than 10.
When the base is 2, this shows that a geometrically distributed random variable can be written as a sum of independent random variables whose probability distributions are indecomposable. Golomb coding is the optimal prefix code for the geometric discrete distribution.
Related distributions
The sum of r independent geometric random variables with parameter p is a negative binomial random variable with parameters r and p . The geometric distribution is a special case of the negative binomial distribution, with r=1 . The geometric distribution is a special case of discrete compound Poisson distribution. The minimum of n geometric random variables with parameters p_1, \dotsc, p_n is also geometrically distributed with parameter 1 - \prod_{i=1}^n (1-p_i) . Suppose 0 k has a Poisson distribution with expected value r k /k.
Then \sum_{k=1}^\infty k\,X_k has a geometric distribution taking values in \mathbb{N}_0 , with expected value r/(1 − r). The exponential distribution is the continuous analogue of the geometric distribution. Applying the floor function to the exponential distribution with parameter \lambda creates a geometric distribution with parameter p=1-e^{-\lambda} defined over \mathbb{N}_0 . This can be used to generate geometrically distributed random numbers as detailed in § Random variate generation.
If p = 1/n and X is geometrically distributed with parameter p, then the distribution of X/n approaches an exponential distribution with expected value 1 as n → ∞, since \begin{align} \Pr(X/n>a)=\Pr(X>na) & = (1-p)^{na} = \left(1-\frac 1 n \right)^{na} = \left[ \left( 1-\frac 1 n \right)^n \right]^{a} \\ & \to [e^{-1}]^{a} = e^{-a} \text{ as } n\to\infty. \end{align} More generally, if p = λ/n, where λ is a parameter, then as n→ ∞ the distribution of X/n approaches an exponential distribution with rate λ: \Pr(X>nx)=\lim_{n \to \infty}(1-\lambda /n)^{nx}=e^{-\lambda x} therefore the distribution function of X/n converges to 1-e^{-\lambda x} , which is that of an exponential random variable.
The index of dispersion of the geometric distribution is \frac{1}{p} and its coefficient of variation is \frac{1}{\sqrt{1-p}} . The distribution is overdispersed.
Statistical inference
The true parameter p of an unknown geometric distribution can be inferred through estimators and conjugate distributions. Provided they exist, the first l moments of a probability distribution can be estimated from a sample x_1, \dotsc, x_n using the formula m_i = \frac{1}{n} \sum_{j=1}^n x^i_j where m_i is the i th sample moment and 1 \leq i \leq l . Estimating \mathrm{E}(X) with m_1 gives the sample mean, denoted \bar{x} .
Substituting this estimate in the formula for the expected value of a geometric distribution and solving for p gives the estimators \hat{p} = \frac{1}{\bar{x}} and \hat{p} = \frac{1}{\bar{x}+1} when supported on \mathbb{N} and \mathbb{N}_0 respectively. These estimators are biased since \mathrm{E}\left(\frac{1}{\bar{x}}\right) > \frac{1}{\mathrm{E}(\bar{x})} = p as a result of Jensen's inequality. The maximum likelihood estimator of p is the value that maximizes the likelihood function given a sample.
By finding the zero of the derivative of the log-likelihood function when the distribution is defined over \mathbb{N} , the maximum likelihood estimator can be found to be \hat{p} = \frac{1}{\bar{x}} , where \bar{x} is the sample mean. If the domain is \mathbb{N}_0 , then the estimator shifts to \hat{p} = \frac{1}{\bar{x}+1} . As previously discussed in § Method of moments, these estimators are biased.
Regardless of the domain, the bias is equal to b \equiv \operatorname{E}\bigg[\;(\hat p_\mathrm{mle} - p)\;\bigg] = \frac{p\,(1-p)}{n} which yields the bias-corrected maximum likelihood estimator, \hat{p\,}^*_\text{mle} = \hat{p\,}_\text{mle} - \hat{b\,} In Bayesian inference, the parameter p is a random variable from a prior distribution with a posterior distribution calculated using Bayes' theorem after observing samples.
If a beta distribution is chosen as the prior distribution, then the posterior will also be a beta distribution and it is called the conjugate distribution. In particular, if a \mathrm{Beta}(\alpha,\beta) prior is selected, then the posterior, after observing samples k_1, \dotsc, k_n \in \mathbb{N} , is p \sim \mathrm{Beta}\left(\alpha+n,\ \beta+\sum_{i=1}^n (k_i-1)\right). \! Alternatively, if the samples are in \mathbb{N}_0 , the posterior distribution is p \sim \mathrm{Beta}\left(\alpha+n,\beta+\sum_{i=1}^n k_i\right).
Since the expected value of a \mathrm{Beta}(\alpha,\beta) distribution is \frac{\alpha}{\alpha+\beta} , as \alpha and \beta approach zero, the posterior mean approaches its maximum likelihood estimate.
References
Text derived from https://en.wikipedia.org/w/index.php?oldid=1375479653 under CC BY-SA 4.0; modifications have been made.