Lesson plan

Information theory

Information theory is the mathematical study of the quantification, storage, and communication of a particular type of mathematically defined information. The field was established and formalized by Claude Shannon in the 1940s, though early contributions were made in the 1920s through the works of Harry Nyquist and Ralph Hartley. Information theory was initially formed in the context of telecommunication but soon found a wide range of other applications. It is now at the intersection of mathematics, statistics and computer science, and has applications in diverse fields ranging from electrical engineering and physics to neurobiology. As a simple example of the concept, if one flips a fair coin and does not yet know the outcome (heads or tails), then one lacks a certain amount of information. After looking at the coin, one gains information about the outcome. For a fair coin, the probability of either heads or tails is 1/2 and the amount of information is expressed as -\log_2(1/2) = 1 bit of information. A key concept in information theory is entropy. In Shannon's formulation, entropy is equal to the lack of information about an event. In the above coin flip example, the entropy in the case where you don't know the outcome is 1 bit. When you know the outcome after the coin has landed, the entropy is zero because you have gained one bit.…

Back to article

Article summary

Information theory is the mathematical study of the quantification, storage, and communication of a particular type of mathematically defined information. The field was established and formalized by Claude Shannon in the 1940s, though early contributions were made in the 1920s through the works of Harry Nyquist and Ralph Hartley. Information theory was initially formed in the context of telecommunication but soon found a wide range of other applications. It is now at the intersection of mathematics, statistics and computer science, and has applications in diverse fields ranging from electrical engineering and physics to neurobiology. As a simple example of the concept, if one flips a fair coin and does not yet know the outcome (heads or tails), then one lacks a certain amount of information. After looking at the coin, one gains information about the outcome. For a fair coin, the probability of either heads or tails is 1/2 and the amount of information is expressed as -\log_2(1/2) = 1 bit of information. A key concept in information theory is entropy. In Shannon's formulation, entropy is equal to the lack of information about an event. In the above coin flip example, the entropy in the case where you don't know the outcome is 1 bit. When you know the outcome after the coin has landed, the entropy is zero because you have gained one bit.…

Key claims

No claims recorded.

Discussion questions

  1. Summarise the article in your own words. Which single claim would you verify first, and why?
  2. Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
  3. If two sources disagreed about one of these facts, what would you need in order to trust one over the other?

Bibliography

No sources recorded.

Key claims and bibliography are generated from the same tracked claims and sources as the article page.