I am studying the book "elements of information theory"- THOMAS M. COVER JOY A. THOMAS, and when it proves the channel coding theorem, one of the things it states is that all codes "C", are symmetric (refer to link). Could someone care to explain why this the codes construction is considered symmetric? All other previous proofs in the book make complete sense to me, it's just this affirmation that has me lost.

link (screenshot of the book where it states symmetry): https://gyazo.com/1cf69ee16a0c606a7c96044170a2945c

For reference, this is chapter 7 subsection 7 of the book previously mentioned.

Thank you!

  • $\begingroup$ These codes are not themselves symmetric. The construction that produces these codes treats all messages in the same way. This is the symmetry that Cover & Thomas is talking about. $\endgroup$ – Peter Shor Mar 25 at 21:48
  • $\begingroup$ @PeterShor what do you mean by code construction symmetry? Aren't we just randomly generating sequences? And why do they consider the given output independent from the input? Is it because it's a "predicted" input? $\endgroup$ – Diogo Landau Mar 26 at 16:07
  • $\begingroup$ Look at the code construction. Does it say "for the first codeword, do this, and for the second codeword, do that"? No. It treats all the codewords in the same way. This is what the book means by "symmetry". $\endgroup$ – Peter Shor Mar 26 at 16:12
  • $\begingroup$ @PeterShor Thank you!! makes sense! Related to this, why does it say that the input and output are independent? Screenshot: gyazo.com/74f66d9f828896c0c007638fe450822c is it because we are just selecting two random sequences out of the 2^N? $\endgroup$ – Diogo Landau Mar 26 at 17:10
  • $\begingroup$ It doesn't say that the input and output are independent. It defines a new distribution where the input and output have the same marginals as in the original distribution but are independent. This distribution is hypothetical; it is different from the original distribution; and it is used in the proof of the channel coding theorem. $\endgroup$ – Peter Shor Mar 26 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.