Information and Coding Theory Texts

Question

I am coming from a pure mathematics (in analysis) background and am curious to learn some information and coding theory. I am after some recommendations on texts. Due to my personal background I am more inclined to prefer books with a higher level of mathematical rigour, however I still greatly appreciate examples that enable and develop the readers intuition.

Can anyone recommend introductory texts on the topics of information and coding theory that has the above properties?

Answer 1

Maybe not so math oriented but with math rigor:

Elements of Information Theory by Thomas M. Cover, Joy A. Thomas

Essential Coding Theory by Venkatesan Guruswami, Atri Rudra and Madhu Sudan

Answer 2

Both texts in the other answer are great texts, and the Guruswami, Rudra, Sudan book is more based in the TCS approach to coding theory, which may be relevant to the potential reader. The books below have been split into the "mainly information theory" and "mainly coding theory" subsets.

Information Theory:

For mathematical rigour, the following text

Csiszar and Korner: Information Theory: Coding Theorems for Discrete Memoryless Systems (1981)

and its updated second edition

Csiszar and Korner: Information Theory: Coding Theorems for Discrete Memoryless Systems (2011)

has not been surpassed in my opinion. For example have a look at the Network Information Theory chapter (Chapter 3 in the first edition) and compare with Cover and Thomas (which is itself a great book to learn/teach from but not really that rigorous).

Another text which is dated now but is really nice is

Coding Theory:

A modern rigorous text mainly focused on the algebraic approach:

Ling and Xing: Coding Theory

Somewhat dated, but a classic and rigorous:

MacWilliams and Sloane: Theory of Error-Correcting Codes

Finally, a Hybrid text which covers both bases that is rigorous and an excellent textbook is Roman: Coding and Information Theory