conditional independence

situations wherein an observation is irrelevant or redundant when evaluating the certainty of a hypothesis

Uses your system voice; voice and rate in settings.

≈ 14 min read

Cite, export and learn8 citation formats · 8 exports · lesson and quiz

Names in other languages

  • EspañolIndependencia condicional
  • PortuguêsIndependência condicional
  • Englishconditional independence
  • Deutschbedingte Unabhängigkeit
  • УкраїнськаУмовна незалежність
  • فارسیاستقلال شرطی
  • 日本語条件付き独立性
  • 中文条件独立

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

In probability theory, conditional independence describes situations in which an observation is irrelevant or redundant when evaluating the certainty of a hypothesis. It is the opposite of conditional dependence.

Conditional independence is usually formulated in terms of conditional probability, as a special case where the probability of the hypothesis given the uninformative observation is equal to the probability without. If A is the hypothesis, and B and C are observations, conditional independence can be stated as an equality: P(A\mid B,C) = P(A \mid C) where P(A \mid B, C) is the probability of A given both B and C . Since the probability of A given C is the same as the probability of A given both B and C , this equality expresses that B contributes nothing to the certainty of A . In this case, A and B are said to be conditionally independent given C , written symbolically as: (A \perp\!\!\!\perp B \mid C) . The concept of conditional independence is essential to graph-based theories of statistical inference, as it establishes a mathematical relation between a collection of conditional statements and a graphoid.

Conditional independence of events

Let A , B , and C be events. A and B are said to be conditionally independent given C if and only if P(C) > 0 and P(A \mid B, C) = P(A \mid C) . This property is symmetric (more on this below) and often written as (A \perp\!\!\!\perp B \mid C) , which should be read as ((A \perp\!\!\!\perp B) \vert C) . Equivalently, conditional independence may be stated as P(A,B|C) = P(A|C)P(B|C) where P(A,B|C) is the joint probability of A and B given C . This alternate formulation states that A and B are independent events, C . It demonstrates that (A \perp\!\!\!\perp B \mid C) is equivalent to (B \perp\!\!\!\perp A \mid C) . P(A, B \mid C) = P(A \mid C) P(B \mid C) & \iff \frac{P(A, B, C)}{P(C)} = \left(\frac{P(A, C)}{P(C)}\right) \left(\frac{P(B, C)}{P(C)} \right) & \text{definition of conditional probability} \\ & \iff P(A, B, C) = \frac{P(A, C) P(B, C)}{P(C)} & \text{multiply both sides by P(C)} \\ & \iff \frac{P(A, B, C)}{P(B, C)}= \frac{P(A, C)}{P(C)} & \text{divide both sides by P(B, C)} \\ & \iff P(A \mid B, C) = P(A \mid C) &\text{definition of conditional probability} \end{align}

Examples

\Pr({\color{red}R}, {\color{blue}B} \mid {\color{gold}Y}) = \Pr({\color{red}R} \mid {\color{gold}Y})\Pr({\color{blue}B} \mid {\color{gold}Y}) but not conditionally independent given \left[ \text{not }{\color{gold}Y}\right] because: \Pr({\color{red}R}, {\color{blue}B} \mid \text{not } {\color{gold}Y}) \not= \Pr({\color{red}R} \mid \text{not } {\color{gold}Y})\Pr({\color{blue}B} \mid \text{not } {\color{gold}Y})

Height and vocabulary

Height and vocabulary are dependent since very small people tend to be children, known for their more basic vocabularies. But knowing that two people are 19 years old (i.e., conditional on age) there is no reason to think that one person's vocabulary is larger if we are told that they are taller.

Conditional independence of random variables

Two discrete random variables X and Y are conditionally independent given a third discrete random variable Z if and only if they are independent in their conditional probability distribution given Z . That is, X and Y are conditionally independent given Z if and only if, given any value of Z , the probability distribution of X is the same for all values of Y and the probability distribution of Y is the same for all values of X . Formally: where F_{X,Y\,\mid\,Z\,=\,z}(x,y)=\Pr(X \leq x, Y \leq y \mid Z=z) is the conditional cumulative distribution function of X and Y given Z . Two events R and B are conditionally independent given a σ-algebra \Sigma if \Pr(R, B \mid \Sigma) = \Pr(R \mid \Sigma)\Pr(B \mid \Sigma) \text{ a.s.} where \Pr(A \mid \Sigma) denotes the conditional expectation of the indicator function of the event A , \chi_A , given the sigma algebra \Sigma . That is, \Pr(A \mid \Sigma) := \operatorname{E}[\chi_A\mid\Sigma].

Two random variables X and Y are conditionally independent given a σ-algebra \Sigma if the above equation holds for all R in \sigma(X) and B in \sigma(Y) . Two random variables X and Y are conditionally independent given a random variable W if they are independent given σ(W): the σ-algebra generated by W . This is commonly written: X \perp\!\!\!\perp Y \mid W or X \perp Y \mid W This is read " X is independent of Y , given W "; the conditioning applies to the whole statement: "( X is independent of Y ) given W ". (X \perp\!\!\!\perp Y) \mid W This notation extends X \perp\!\!\!\perp Y for " X is independent of Y ." If W assumes a countable set of values, this is equivalent to the conditional independence of X and Y for the events of the form [W=w] . Conditional independence of more than two events, or of more than two random variables, is defined analogously. The following two examples show that X \perp\!\!\!\perp Y neither implies nor is implied by (X \perp\!\!\!\perp Y) \mid W . First, suppose W is 0 with probability 0.5 and 1 otherwise.

When W = 0 take X and Y to be independent, each having the value 0 with probability 0.99 and the value 1 otherwise. When W=1 , X and Y are again independent, but this time they take the value 1 with probability 0.99. Then (X \perp\!\!\!\perp Y) \mid W . But X and Y are dependent, because Pr(X = 0)  0.5.

For the second example, suppose X \perp\!\!\!\perp Y , each taking the values 0 and 1 with probability 0.5. Let W be the product X \cdot Y . Then when W=0 , Pr(X = 0) = 2/3, but Pr(X = 0|Y = 0) = 1/2, so (X \perp\!\!\!\perp Y) \mid W is false. This is also an example of Explaining Away.

See Kevin Murphy's tutorial where X and Y take the values "brainy" and "sporty".

Conditional independence of random vectors

Two random vectors \mathbf{X}=(X_1,\ldots,X_l)^{\mathrm T} and \mathbf{Y}=(Y_1,\ldots,Y_m)^{\mathrm T} are conditionally independent given a third random vector \mathbf{Z}=(Z_1,\ldots,Z_n)^{\mathrm T} if and only if they are independent in their conditional cumulative distribution given \mathbf{Z} . Formally: {{Equation box 1 |indent = |title= |equation = {{NumBlk|| (\mathbf{X} \perp\!\!\!\perp \mathbf{Y}) \mid \mathbf{Z} \quad \iff \quad F_{\mathbf{X},\mathbf{Y}|\mathbf{Z}=\mathbf{z}}(\mathbf{x},\mathbf{y}) = F_{\mathbf{X}\,\mid\,\mathbf{Z}\,=\,\mathbf{z}}(\mathbf{x}) \cdot F_{\mathbf{Y}\,\mid\,\mathbf{Z}\,=\,\mathbf{z}}(\mathbf{y}) \quad \text{for all } \mathbf{x},\mathbf{y},\mathbf{z} | }} |cellpadding= 6 |border |border colour = #0073CF |background colour=#F5FFFA}} where \mathbf{x}=(x_1,\ldots,x_l)^{\mathrm T} , \mathbf{y}=(y_1,\ldots,y_m)^{\mathrm T} and \mathbf{z}=(z_1,\ldots,z_n)^{\mathrm T} and the conditional cumulative distributions are defined as follows. \begin{align} F_{\mathbf{X},\mathbf{Y}\,\mid\,\mathbf{Z}\,=\,\mathbf{z}}(\mathbf{x},\mathbf{y}) &= \Pr(X_1 \leq x_1,\ldots,X_l \leq x_l,

Y_1 \leq y_1,\ldots,Y_m \leq y_m \mid Z_1=z_1,\ldots,Z_n=z_n) \\[6pt] F_{\mathbf{X}\,\mid\,\mathbf{Z}\,=\,\mathbf{z}}(\mathbf{x}) &= \Pr(X_1 \leq x_1,\ldots,X_l \leq x_l \mid Z_1=z_1,\ldots,Z_n=z_n) \\[6pt] F_{\mathbf{Y}\,\mid\,\mathbf{Z}\,=\,\mathbf{z}}(\mathbf{y}) &= \Pr(Y_1 \leq y_1,\ldots,Y_m \leq y_m \mid Z_1=z_1,\ldots,Z_n=z_n) \end{align}

Uses in Bayesian inference

Let p be the proportion of voters who will vote "yes" in an upcoming referendum. In taking an opinion poll, one chooses n voters randomly from the population. For i = 1, ..., n, let X i  = 1 or 0 corresponding, respectively, to whether or not the ith chosen voter will or will not vote "yes".

In a frequentist approach to statistical inference one would not attribute any probability distribution to p (unless the probabilities could be somehow interpreted as relative frequencies of occurrence of some event or as proportions of some population) and one would say that X 1 , ..., X n are independent random variables. By contrast, in a Bayesian approach to statistical inference, one would assign a probability distribution to p regardless of the non-existence of any such "frequency" interpretation, and one would construe the probabilities as degrees of belief that p is in any interval to which a probability is assigned.

In that model, the random variables X 1 , ..., X n are not independent, but they are conditionally independent given the value of p. In particular, if a large number of the Xs are observed to be equal to 1, that would imply a high conditional probability, given that observation, that p is near 1, and thus a high conditional probability, given that observation, that the next X to be observed will be equal to 1.

Rules of conditional independence

A set of rules governing statements of conditional independence have been derived from the basic definition. These rules were termed "Graphoid Axioms" by Pearl and Paz, because they hold in graphs, where X \perp\!\!\!\perp A\mid B is interpreted to mean: "All paths from X to A are intercepted by the set B".

Symmetry

X \perp\!\!\!\perp Y \mid Z \quad \Leftrightarrow \quad Y \perp\!\!\!\perp X \mid Z Proof: From the definition of conditional independence, X \perp\!\!\!\perp Y \mid Z \quad \Leftrightarrow \quad P(X, Y \mid Z) = P(X \mid Z) P(Y \mid Z) \quad \Leftrightarrow \quad Y \perp\!\!\!\perp X \mid Z

Decomposition

X \perp\!\!\!\perp Y \mid Z \quad \Rightarrow \quad h(X) \perp\!\!\!\perp Y \mid Z Proof From the definition of conditional independence, we seek to show that: X \perp\!\!\!\perp Y \mid Z \quad \Rightarrow \quad P(h(X), Y \mid Z) = P(h(X) \mid Z) P(Y \mid Z) . The left side of this equality is: P(h(X)=a, Y=y \mid Z=z) = \sum_{X \colon h(X)=a} P(X=x, Y=y \mid Z=z) , where the expression on the right side of this equality is the summation over X such that h(X)=a of the conditional probability of X, Y on Z . Further decomposing, \begin{align} \sum_{X \colon h(X)=a} P(X=x,

Y=y \mid Z=z) =& \sum_{X \colon h(X)=a} P(X=x \mid Z=z) P(Y=y \mid Z=z) \\ =& P(Y=y \mid Z=z) \sum_{X \colon h(X)=a} P(X=x \mid Z=z) \\ =& P(Y \mid Z) P (h(X) \mid Z) \end{align} . Special cases of this property include (X,

W) \perp\!\!\!\perp Y \mid Z \quad \Rightarrow \quad X \perp\!\!\!\perp Y \mid Z Proof: Let us define A = (X,W) and h(\cdot) be an 'extraction' function h(X,W) = X . Then: \begin{align} (X,W) \perp\!\!\!\perp Y \mid Z \quad &\Leftrightarrow \quad A \perp\!\!\!\perp Y \mid Z \\ &\Rightarrow \quad h(A) \perp\!\!\!\perp Y \mid Z \quad &\text{Decomposition} \\ &\Leftrightarrow \quad X \perp\!\!\!\perp Y \mid Z \end{align} X \perp\!\!\!\perp (Y,

W) \mid Z \quad \Rightarrow \quad X \perp\!\!\!\perp Y \mid Z Proof: Let us define V = (Y,W) and h(\cdot) be again an 'extraction' function h(Y,W) = Y . Then: \begin{align} X \perp\!\!\!\perp (Y,W) \mid Z \quad &\Leftrightarrow \quad X \perp\!\!\!\perp V \mid Z \\ &\Leftrightarrow \quad V \perp\!\!\!\perp X \mid Z \quad &\text{Symmetry} \\ &\Rightarrow \quad h(V) \perp\!\!\!\perp X \mid Z \quad &\text{Decomposition} \\ &\Leftrightarrow \quad Y \perp\!\!\!\perp X \mid Z \\ &\Leftrightarrow \quad X \perp\!\!\!\perp Y \mid Z \quad &\text{Symmetry} \end{align}

Weak union

X \perp\!\!\!\perp Y \mid Z \quad \Rightarrow \quad X \perp\!\!\!\perp Y \mid (Z, h(X)) Proof: Given X \perp\!\!\!\perp Y \mid Z , we aim to show \begin{align} X \perp\!\!\!\perp Y \mid (Z, h(X)) \quad &\Leftrightarrow \quad X \perp\!\!\!\perp Y \mid U \quad &\text{where} \quad U = (Z, h(X)) \\ &\Leftrightarrow \quad Y \perp\!\!\!\perp X \mid U \quad &\text{Symmetry} \\ &\Leftrightarrow \quad P(Y\mid X, U) = P(Y\mid U) \\ &\Leftrightarrow \quad P(Y \mid X, Z, h(X)) = P(Y \mid Z, h(X)) \end{align} . We begin with the left side of the equation \begin{align} P(Y \mid X, Z, h(X)) &= P(Y \mid X, Z) \\ &= P(Y \mid Z) &\text{Since by symmetry } Y \perp\!\!\!\perp X \mid Z \end{align} .

From the given condition \begin{align} X \perp\!\!\!\perp Y \mid Z \quad &\Rightarrow \quad h(X) \perp\!\!\!\perp Y \mid Z \quad &\text{Decomposition} \\ &\Leftrightarrow \quad Y \perp\!\!\!\perp h(X) \mid Z \quad &\text{Symmetry} \\ &\Rightarrow \quad P(Y \mid Z, h(X)) = P(Y \mid Z) \end{align} . Thus P(Y \mid X, Z, h(X)) = P(Y \mid Z, h(X)) , so we have shown that X \perp\!\!\!\perp Y \mid (Z, h(X)) . Special Cases: Some textbooks present the property as X \perp\!\!\!\perp (Y, W) \mid Z \quad \Rightarrow \quad X \perp\!\!\!\perp Y \mid (Z, W) . (X,W) \perp\!\!\!\perp Y \mid Z \quad \Rightarrow \quad X \perp\!\!\!\perp Y \mid (Z,W) . Both versions can be shown to follow from the weak union property given initially via the same method as in the decomposition section above.

Contraction

\left.\begin{align} X \perp\!\!\!\perp A \mid B \\ X \perp\!\!\!\perp B \end{align}\right\}\text{ and } \quad \Rightarrow \quad X \perp\!\!\!\perp A,B Proof This property can be proved by noticing \Pr(X\mid A,B) = \Pr(X\mid B) = \Pr(X) , each equality of which is asserted by X \perp\!\!\!\perp A \mid B and X \perp\!\!\!\perp B , respectively.

Intersection

For strictly positive probability distributions, the following also holds: \left.\begin{align} X \perp\!\!\!\perp Y \mid Z, W\\ X \perp\!\!\!\perp W \mid Z, Y \end{align}\right\}\text{ and } \quad \Rightarrow \quad X \perp\!\!\!\perp W, Y \mid Z Proof By assumption: P(X|Z, W, Y) = P(X|Z, W) \land P(X|Z, W, Y) = P(X|Z, Y) \implies P(X|Z, Y) = P(X|Z, W) Using this equality, together with the Law of total probability applied to P(X|Z) : \begin{align} P(X|Z) &= \sum_{w \in W} P(X|Z, W=w)P(W=w|Z) \\[4pt] &= \sum_{w \in W} P(X|Y, Z)P(W=w|Z) \\[4pt] &= P(X|Z, Y) \sum_{w \in W} P(W=w|Z) \\[4pt] &= P(X|Z, Y) \end{align} Since P(X|Z, W, Y) = P(X|Z, Y) and P(X|Z, Y) = P(X|Z) , it follows that P(X|Z, W, Y) = P(X|Z) \iff X \perp\!\!\!\perp Y,W | Z . Technical note: since these implications hold for any probability space, they will still hold if one considers a sub-universe by conditioning everything on another variable, say K. For example, X \perp\!\!\!\perp Y \Rightarrow Y \perp\!\!\!\perp X would also mean that X \perp\!\!\!\perp Y \mid K \Rightarrow Y \perp\!\!\!\perp X \mid K .

See also

Graphoid Conditional dependence de Finetti's theorem Conditional expectation

Text derived from https://en.wikipedia.org/w/index.php?oldid=1356071893 under CC BY-SA 4.0; modifications have been made.

Words in each sectionConditional independenc…Conditional independence of random variablesConditional independence of random variables: 517 words517DecompositionDecompositionDecomposition: 374 words374Weak unionWeak unionWeak union: 292 words292Conditional independenc…Conditional independence of random vectorsConditional independence of random vectors: 285 words285Conditional independenc…Conditional independence of eventsConditional independence of events: 238 words238Uses in Bayesian infere…Uses in Bayesian inferenceUses in Bayesian inference: 233 words233IntersectionIntersectionIntersection: 233 words233ExamplesExamplesExamples: 132 words132ContractionContractionContraction: 64 words64Rules of conditional in…Rules of conditional independenceRules of conditional independence: 58 words58SymmetrySymmetrySymmetry: 53 words53See alsoSee alsoSee also: 31 words31
Words in each section. Measured from the stored article body; sections below the minimum are not plotted.

Provenance

snapshot conditional-independence-587870-672b6773a9da4c5f · hash 672b6773a9da4c5fa161b4a94900b6e3735a002bc8218b3f9510e40a44a1066c · captured 2026-05-25 · ledger · version history

Cite this page as: encyclopegia.org, “conditional independence”, snapshot conditional-independence-587870-672b6773a9da4c5f. https://encyclopegia.org/a/conditional-independence

Sources cited in this revision

No external citation was extracted for this revision. That is a statement about the source, not a claim that none exists.