State-dependent information
≈ 2 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In information theory, state-dependent information is the generic name given to the family of state-dependent measures that in expectation converge to the mutual information. State-dependent informations often appear in neuroscience applications. Let X and Y be random variables and y be a state within Y . The state-dependent information between a random variable X and a state Y=y is written as I( X ; Y = y) . There are currently three known varieties of state-dependent information: specific-surprise, specific-information, and state-specific-information.
In information theory, state-dependent information is the generic name given to the family of state-dependent measures that in expectation converge to the mutual information. State-dependent informations often appear in neuroscience applications. Let X and Y be random variables and y be a state within Y . The state-dependent information between a random variable X and a state Y=y is written as I( X ; Y = y) .
There are currently three known varieties of state-dependent information: specific-surprise, specific-information, and state-specific-information. \mathrm{I_{ss}}(X;Y=y) \equiv \mathrm{D_{KL}}[ P_{X|y} \parallel P_{X} ] . As a special case of the chain-rule for Kullback-Liebler divergerences, specific-surprise follows the chain-rule for variables. Using Z as a random variable, this is specifically, \mathrm{I_{ss}}(XZ;Y=y) = \mathrm{I_{ss}}(X;Y=y) + \mathrm{I_{ss}}(Z;Y=y|X) .
Intuitively, specific-surprise is thought of as “how much did my beliefs about X change upon learning that Y=y ”? Which is zero when there’s no change. It is nonnegative. Specific-surprise has also been called “Bayesian Surprise”. \mathrm{I_{si}}(X;Y=y) \equiv \mathrm{H}(X) - \mathrm{H}(X|Y=y) . Specific-information follows the chain-rule for states. Using a state z \in Z as a state of random variable Z , this is specifically, \mathrm{I_{si}}(X;YZ=yz) = \mathrm{I_{si}}(X;Y=y) + \mathrm{I_{si}}(X;Z=z|Y=y) .
Specific-information is interpreted as "how did the uncertainty about X change upon learning Y=y ?" This can be in the positive or negative. When X follows a uniform distribution, the \mathrm{I_{ss}} and \mathrm{I_{si}} are equivalent.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1334358963 under CC BY-SA 4.0; modifications have been made.