exchangeable random variables
≈ 6 min read
Names in other languages
- Deutschaustauschbare Familie von Zufallsvariablen
- Españolvariables aleatorias intercambiables
- Українськавзаємозамінні випадкові змінні
- Englishexchangeable random variables
- فارسیمتغیرهای تصادفی تعویض پذیر
- Nederlandsuitwisselbaarheid
- 한국어교환 가능 확률 변수족
- 日本語交換可能な確率変数
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In statistics, an exchangeable sequence of random variables (also sometimes interchangeable) is a sequence X 1 , X 2 , X 3 , ... (which may be finitely or infinitely long) whose joint probability distribution does not change when the positions in the sequence in which finitely many of them appear are altered. In other words, the joint distribution is invariant to finite permutation. Thus, for example the sequences : X_1, X_2, X_3, X_4, X_5, X_6 \quad \text{ and } \quad X_3, X_6, X_1, X_5, X_2, X_4 both have the same joint probability distribution. It is closely related to the use of independent and identically distributed random variables in statistical models. Exchangeable sequences of random variables arise in cases of simple random sampling.
In statistics, an exchangeable sequence of random variables (also sometimes interchangeable) is a sequence X 1 , X 2 , X 3 , ... (which may be finitely or infinitely long) whose joint probability distribution does not change when the positions in the sequence in which finitely many of them appear are altered. In other words, the joint distribution is invariant to finite permutation. Thus, for example the sequences X_1, X_2, X_3, X_4, X_5, X_6 \quad \text{ and } \quad X_3, X_6, X_1, X_5, X_2, X_4 both have the same joint probability distribution. It is closely related to the use of independent and identically distributed random variables in statistical models. Exchangeable sequences of random variables arise in cases of simple random sampling. The representation theorem: This statement is based on the presentation in O’Neill (2009) in the references below. Given an infinite sequence of random variables \mathbf{X}=(X_1,X_2,X_3,\ldots) we define the limiting empirical distribution function F_\mathbf{X} by F_\mathbf{X}(x) = \lim_{n\to\infty} \frac{1}{n} \sum_{i=1}^n I(X_i \le x). (This is the Cesàro limit of the indicator functions. In cases where the Cesàro limit does not exist this function can actually be defined as the Banach limit of the indicator functions, which is an extension of this limit. This latter limit always exists for sums of indicator functions, so that the empirical distribution is always well-defined.) This means that for any vector of random variables in the sequence we have joint distribution function given by \Pr (X_1 \le x_1,X_2 \le x_2,\ldots,X_n \le x_n) = \int \prod_{i=1}^n F_\mathbf{X}(x_i)\,dP(F_\mathbf{X}). If the distribution function F_\mathbf{X} is indexed by another parameter \theta then (with densities appropriately defined) we have p_{X_1,\ldots,X_n}(x_1,\ldots,x_n) = \int \prod_{i=1}^n p_{X_i}(x_i\mid\theta)\,dP(\theta). These equations show the joint distribution or density characterised as a mixture distribution based on the underlying limiting empirical distribution (or a parameter indexing this distribution). Note that not all finite exchangeable sequences are mixtures of i.i.d.. To see this, consider sampling without replacement from a finite set until no elements are left. The resulting sequence is exchangeable, but not a mixture of i.i.d.. Indeed, conditioned on all other elements in the sequence, the remaining element is known. \operatorname{cov} (X_i,X_j) = \operatorname{var} (\operatorname{E}(X_i\mid F_\mathbf{X})) = \operatorname{var} (\operatorname{E}(X_i\mid\theta)) \ge 0 \quad\text{for }i \ne j. Covariance for exchangeable sequences (finite): If X_1,X_2,\ldots,X_n is exchangeable with \sigma^2 = \operatorname{var} (X_i) , then \operatorname{cov} (X_i,X_j) \ge - \frac{\sigma^2}{n-1} \quad\text{for }i \ne j. The finite sequence result may be proved as follows. Using the fact that the values are exchangeable, we have \begin{align} 0 & \le \operatorname{var}(X_1 + \cdots + X_n) \\ & = \operatorname{var}(X_1) + \cdots + \operatorname{var}(X_n) + \underbrace{\operatorname{cov}(X_1,X_2) + \cdots\quad{}}_\text{all ordered pairs} \\ & = n\sigma^2 + n(n-1)\operatorname{cov}(X_1,X_2). \end{align} We can then solve the inequality for the covariance yielding the stated lower bound. The non-negativity of the covariance for the infinite sequence can then be obtained as a limiting result from this finite sequence result. Equality of the lower bound for finite sequences is achieved in a simple urn model: An urn contains 1 red marble and n − 1 green marbles, and these are sampled without replacement until the urn is empty. Let X i = 1 if the red marble is drawn on the i-th trial and 0 otherwise. A finite sequence that achieves the lower covariance bound cannot be extended to a longer exchangeable sequence. Suppose an urn contains n red and m blue marbles. Suppose marbles are drawn without replacement until the urn is empty. Let X_i be the indicator random variable of the event that the i -th marble drawn is red. Then \left\{ X_i \right\}_{i=1, \dots, n+m} is an exchangeable sequence. This sequence cannot be extended to any longer exchangeable sequence. Suppose an urn contains n red and m blue marbles. Further suppose a marble is drawn from the urn and then replaced, with an extra marble of the same colour. Let X_i be the indicator random variable of the event that the i -th marble drawn is red. Then \left\{ X_i \right\}_{i\in \N} is an exchangeable sequence. This model is called Polya's urn. Let (X, Y) have a bivariate normal distribution with parameters \mu = 0 , \sigma_x = \sigma_y = 1 and an arbitrary correlation coefficient \rho\in (-1, 1) . The random variables X and Y are then exchangeable, but independent only if \rho=0 . The density function is p(x, y) = p(y, x) \propto \exp\left[-\frac{1}{2(1-\rho^2)}(x^2+y^2-2\rho xy)\right]. The von Neumann extractor is a randomness extractor that depends on exchangeability: it gives a method to take an exchangeable sequence of 0s and 1s (Bernoulli trials), with some probability p of 0 and q=1-p of 1, and produce a (shorter) exchangeable sequence of 0s and 1s with probability 1/2. Partition the sequence into non-overlapping pairs: if the two elements of the pair are equal (00 or 11), discard it; if the two elements of the pair are unequal (01 or 10), keep the first. This yields a sequence of Bernoulli trials with p=1/2, as, by exchangeability, the odds of a given pair being 01 or 10 are equal. Exchangeable random variables arise in the study of U statistics, particularly in the Hoeffding decomposition. Exchangeability is a key assumption of the distribution-free inference method of conformal prediction.
See also
De Finetti theorem Hewitt-Savage zero-one law Resampling , statistical tests based on exchanging between groups
Text derived from https://en.wikipedia.org/w/index.php?oldid=1334892634 under CC BY-SA 4.0; modifications have been made.