I am classically trained in physics, however I have been interested in the use of information theory in studying some classical systems.
As someone who is somewhat unfamiliar with the language of the subject, the entry level explanations are often useless and more specific ones I frankly am unable to totally grasp.
I have been interested in the Chomsky-Schutzenberger enumeration theorem and its implications (in an analogue classical system), however the definition of “word length” (say, from Wikipedia) is lost on me: the expression for “the number of words of length k in L” is the “intersection of a context free L and”... what exactly is sigma in k-powers?
Furthermore, how is word length understood from the context of information theory (shannon entropies, message lengths, lossless encoding, etc.) and what are the implications of context free language and grammar in computer sciences?
I appreciate any responses!