I have started Information Theory classes just recently and was wondering what would be a standard book to purchase. I know I can go for basic introductory books but I also like to purchase standard books that I can use throughout my career for future reference purposes. I get some threads in forums about introductory books or basic books on information theory, but I can't find any about standard books on information theory. P.S.:By standard book I mean like Cormen is for Algorithms


1 Answer 1


This is a list of recommended books, videos and web sites copied from the Further Readings section of my book on information theory (given at the end of this post).

Applebaum D (2008). Probability and Information: An Integrated Approach. A thorough introduction to information theory, which strikes a good balance between intuitive and technical explanations.

Avery J (2012). Information Theory and Evolution. An engaging account of how information theory is relevant to a wide range of natural and man-made systems, including evolution, physics, culture and genetics. Includes interesting background stories on the development of ideas within these different disciplines.

Baeyer HV (2005). Information: The New Language of Science Erudite, wide-ranging, and insightful account of information theory. Contains no equations, which makes it very readable.

Cover T and Thomas J (1991). Elements of Information Theory. Comprehensive, and highly technical, with historical notes and an equation summary at the end of each chapter.

Ghahramani Z (2002). Information Theory. Encyclopedia of Cognitive Science. An excellent, brief overview of information.

Gleick J (2012). The Information. An informal introduction to the history of ideas and people associated with information theory.

Guizzo EM (2003). The Essential Message: Claude Shannon and the Making of Information Theory. Master’s Thesis, Massachusetts Institute of Technology. One of the few accounts of Shannon’s role in the development of information theory. See http://dspace.mit.edu/ bitstream/handle/1721.1/39429/54526133.pdf.

Laughlin, SB (2006). The Hungry Eye: Energy, Information and Retinal Function, Excellent lecture on the energy cost of Shannon information in eyes. See http://www.crsltd.com/guest-talks/ crs-guest-lecturers/simon-laughlin.

MacKay DJC (2003). Information Theory, Inference, and Learning Algorithms. The modern classic on information theory. A very readable text that roams far and wide over many topics. The book’s web site (below) also has a link to an excellent series of video lectures by MacKay. Available free online at http://www.inference.phy. cam.ac.uk/mackay/itila/.

Pierce JR (1980). An Introduction to Information Theory: Symbols, Signals and Noise. Second Edition. Pierce writes with an informal, tutorial style of writing, but does not flinch from presenting the fundamental theorems of information theory. This book provides a good balance between words and equations.

Reza FM (1961). An Introduction to Information Theory. A more comprehensive and mathematically rigorous book than Pierce’s book, it should be read only after first reading Pierce’s more informal text.

Seife C (2007). Decoding the Universe: How the New Science of Information Is Explaining Everything in the Cosmos, From Our Brains to Black Holes. A lucid and engaging account of the relationship between information, thermodynamic entropy and quantum computing. Highly recommended.

Shannon CE and Weaver W (1949). The Mathematical Theory of Communication. University of Illinois Press. A surprisingly accessible book, written in an era when information theory was known only to a privileged few. This book can be downloaded from http://cm. bell-labs.com/cm/ms/what/shannonday/paper.html

For the complete novice, the videos at the online Kahn Academy provide an excellent introduction.

Additionally, the online Scholarpedia web page by Latham and Rudi provides a lucid technical account of mutual information: http://www.scholarpedia.org/article/Mutual_information.

Finally, some historical perspective is provided in a long interview with Shannon conducted in 1982: http://www.ieeeghn.org/wiki/ index.php/Oral-History:Claude_E._Shannon.

Finally, my book is called Information Theory: A Tutorial Introduction (2015). Chapter 1 can be downloaded from here: http://jim-stone.staff.shef.ac.uk/BookInfoTheory/InfoTheoryBookMain.html

enter image description here

  • $\begingroup$ The books of Applebaum and Stone (you) were sufficient for me. $\endgroup$ Commented Mar 17, 2020 at 5:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.