Binomial test

Uses your system voice; voice and rate in settings.

Snapshot

≈ 7 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

Binomial test is an exact test of the statistical significance of deviations from a theoretically expected distribution of observations into two categories using sample data.

Binomial test is an exact test of the statistical significance of deviations from a theoretically expected distribution of observations into two categories using sample data.

Usage

A binomial test is a statistical hypothesis test used to determine whether the proportion of successes in a sample differs from an expected proportion in a binomial distribution. It is useful for situations when there are two possible outcomes (e.g., success/failure, yes/no, heads/tails), i.e., where repeated experiments produce binary data.

If one assumes an underlying probability \pi_0 between 0 and 1, the null hypothesis is H_0\colon\pi=\pi_0 For a sample of size n , we would expect n\pi_0 successes. The formula of the binomial distribution gives the probability of those n samples instead producing k successes: \Pr(X=k)=\binom{n}{k}\pi_0^k(1-\pi_0)^{n-k} Suppose that we want to test the alternative hypothesis H_{A}\colon\pi i.e., we suspect that the actual probability of success is lower than \pi_0 .

Then the p -value of our experiment would be computed using a one-tailed test; specifically, we compute the probability of seeing an outcome as extreme as, or more extreme (i.e., less likely), than k (where k is defined as the number of successes in the n trials of our experiment): p = \sum_{i=0}^k\Pr(X=i)=\sum_{i=0}^k\binom{n}{i}\pi_0^i(1-\pi_0)^{n-i} An analogous computation can be done if we're testing if \pi>\pi_0 using the summation of the range from k to n instead. Calculating a p -value for a two-tailed test is slightly more complicated, since a binomial distribution isn't symmetric if \pi_0\neq 0.5 .

This means that we can't just double the p -value from the one-tailed test. Recall that we want to consider events that are as extreme, or more extreme, than the one we've seen, so we should consider the probability that we would see an event that is as, or less, likely than X=k . Let \mathcal{I}=\{i\colon\Pr(X=i)\leq \Pr(X=k)\} denote all such events.

Then the two-tailed p -value is calculated as, p = \sum_{i\in\mathcal{I}}\Pr(X=i)=\sum_{i\in\mathcal{I}}\binom{n}{i}\pi_0^i(1-\pi_0)^{n-i}

Common use

One common use of the binomial test is the case where the null hypothesizes that two categories occur with equal frequency ( H_0\colon\pi=0.5 ), such as a coin toss. Tables are widely available to give the significance observed numbers of observations in the categories for this case. However, as the example below shows, the binomial test is not restricted to this case.

When there are more than two categories, and an exact test is required, the multinomial test, based on the multinomial distribution, must be used instead of the binomial test. Most common measures of effect size for Binomial tests are Cohen's h or Cohen's g. Z=\frac{k-n\pi}{\sqrt{n\pi(1-\pi)}} where k is the number of successes observed in a sample of size n and \pi is the probability of success according to the null hypothesis.

An improvement on this approximation is possible by introducing a continuity correction: Z=\frac{k-n\pi\pm \frac{1}{2}}{\sqrt{n\pi(1-\pi)}} For very large n , this continuity correction will be unimportant, but for intermediate values, where the exact binomial test doesn't work, it will yield a substantially more accurate result. In notation in terms of a measured sample proportion \hat{p} , null hypothesis for the proportion p_0 , and sample size n , where \hat{p}=k/n and p_0=\pi , one may rearrange and write the z-test above as Z=\frac{ \hat{p}-p_0 } { \sqrt{ \frac{p_0(1-p_0)}{n} } } by dividing by n in both numerator and denominator, which is a form that may be more familiar to some readers.

Example

Suppose we have a board game that depends on the roll of one die and attaches special importance to rolling a 6. In a particular game, the die is rolled 235 times, and 6 comes up 51 times. If the die is fair, we would expect 6 to come up 235\times1/6 = 39.17 times.

We have now observed that the number of 6s is higher than what we would expect on average by pure chance had the die been a fair one. But, is the number significantly high enough for us to conclude anything about the fairness of the die? This question can be answered by the binomial test.

Our null hypothesis would be that the die is fair (probability of each number coming up on the die is 1/6). To find an answer to this question using the binomial test, we use the binomial distribution B(N=235, p=1/6) with pmf f(k,n,p) = \Pr(k;n,p) = \Pr(X = k) = \binom{n}{k}p^k(1-p)^{n-k} .

As we have observed a value greater than the expected value, we could consider the probability of observing 51 6s or higher under the null, which would constitute a one-tailed test (here we are basically testing whether this die is biased towards generating more 6s than expected). In order to calculate the probability of 51 or more 6s in a sample of 235 under the null hypothesis we add up the probabilities of getting exactly 51 6s, exactly 52 6s, and so on up to probability of getting exactly 235 6s: \sum_{i=51}^{235} {235\choose i}p^i(1-p)^{235-i} = 0.02654 If we have a significance level of 5%, then this result (0.02654 binom.test(51, 235, 1/6, alternative = "less") (one-tailed test) binom.test(51, 235, 1/6, alternative = "greater") (one-tailed test) binom.test(51, 235, 1/6, alternative = "two.sided") (two-tailed test) In Java using the Apache Commons library: new BinomialTest().binomialTest(235, 51, 1.0 / 6, AlternativeHypothesis.LESS_THAN) (one-tailed test) new BinomialTest().binomialTest(235, 51, 1.0 / 6, AlternativeHypothesis.GREATER_THAN) (one-tailed test) new BinomialTest().binomialTest(235, 51, 1.0 / 6, AlternativeHypothesis.TWO_SIDED) (two-tailed test) In SAS the test is available in the Frequency procedure PROC FREQ DATA=DiceRoll ; TABLES Roll / BINOMIAL (P=0.166667) ALPHA=0.05 ; EXACT BINOMIAL ; WEIGHT Freq ; RUN; In SPSS the test can be utilized through the menu Analyze > Nonparametric test > Binomial npar tests /binomial (.5) = node1 node2.

In Python, use SciPy's binomtest: scipy.stats.binomtest(51, 235, 1.0/6, alternative="greater") (one-tailed test) scipy.stats.binomtest(51, 235, 1.0/6, alternative="two-sided") (two-tailed test) In MATLAB, use myBinomTest, which is available via Mathworks' community File Exchange website. myBinomTest will directly calculate the p-value for the observations given the hypothesized probability of a success. [pout]=myBinomTest(51, 235, 1/6) (generally two-tailed, but can optionally perform a one-tailed test).

In Stata, use bitest. In Microsoft Excel, use Binom.Dist. The function takes parameters (Number of successes, Trials, Probability of Success, Cumulative).

The "Cumulative" parameter takes a Boolean True or False, with True giving the Cumulative probability of finding this many successes (a left-tailed test), and False the exact probability of finding this many successes.

See also

p-value Cohen's g Cohen's h Lady tasting tea experiment

Binomial Probability Calculator

Text derived from https://en.wikipedia.org/w/index.php?oldid=1341295792 under CC BY-SA 4.0; modifications have been made.

Sources cited in this revision

No external citation was extracted for this revision. That is a statement about the source, not a claim that none exists.