Lesson plan
Entropic vector
The entropic vector or entropic function is a concept arising in information theory. It represents the possible values of Shannon's information entropy that subsets of one set of random variables may take. Understanding which vectors are entropic is a way to represent all possible inequalities between entropies of various subsets. For example, for any two random variables X_1,X_2 , their joint entropy (the entropy of the random variable representing the pair X_1,X_2 ) is at most the sum of the entropies of X_1 and of X_2 : : H(X_1,X_2) \leq H(X_1) + H(X_2) Other information-theoretic measures such as conditional information, mutual information, or total correlation can be expressed in terms of joint entropy and are thus related by the corresponding inequalities. Many inequalities satisfied by entropic vectors can be derived as linear combinations of a few basic ones, called Shannon-type inequalities. However, it has been proven that already for n=4 variables, no finite set of linear inequalities is sufficient to characterize all entropic vectors.
Article summary
The entropic vector or entropic function is a concept arising in information theory. It represents the possible values of Shannon's information entropy that subsets of one set of random variables may take. Understanding which vectors are entropic is a way to represent all possible inequalities between entropies of various subsets. For example, for any two random variables X_1,X_2 , their joint entropy (the entropy of the random variable representing the pair X_1,X_2 ) is at most the sum of the entropies of X_1 and of X_2 : : H(X_1,X_2) \leq H(X_1) + H(X_2) Other information-theoretic measures such as conditional information, mutual information, or total correlation can be expressed in terms of joint entropy and are thus related by the corresponding inequalities. Many inequalities satisfied by entropic vectors can be derived as linear combinations of a few basic ones, called Shannon-type inequalities. However, it has been proven that already for n=4 variables, no finite set of linear inequalities is sufficient to characterize all entropic vectors.
Key claims
No claims recorded.
Discussion questions
- Summarise the article in your own words. Which single claim would you verify first, and why?
- Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
- If two sources disagreed about one of these facts, what would you need in order to trust one over the other?
Bibliography
No sources recorded.
Key claims and bibliography are generated from the same tracked claims and sources as the article page.