# Are Shannon entropy and Boltzmann entropy mutually convertible?

Are Shannon entropy and Boltzmann entropy mutually convertible, much like mass and energy according to Einstein's formula?

-
Nothing theoretical but here is the Boltzmann formula on his grave in Vienna. –  user1614113 Jan 30 '13 at 11:57

In practice the only difference is that Boltzmann entropy deals with a thermodynamical constant $K_B$:

$H = -K_B\sum_{i=1}^{N} P_i log_e\ P_i$

i assume you already know that if $K_B=1$ you have Shannon entropy in a different base. However the conceptual backgrounds are important; probability of events (Shannon) and probability of a particle being in one of $N$ states (Boltzmann) under a thermodynamical context.

There is theoretical work in making the formal link between the two notions of entropy. Take a look at the work of Edwin Jaynes.

http://en.wikipedia.org/wiki/Edwin_Jaynes

-
I would be interested to know whether there could be not only a formal link but sort of a practical link, much like when mass is transformed into energy. As layman I have a little bit "speculative" thought: If in a certain empirical study I have collected data, there is yet no information for me. I get information after I apply, say, a statistical software to the data. But that software must be run on a machine, where thermodynamical processes are certainly involved. So I "guess" that the informational entropy comes somehow from the thermodynamical entropy involved there. –  Mok-Kong Shen Sep 27 '12 at 19:31
I cannot prove the following but i think in the case you mention, one should consider entropy on the first event when the data has been collected and contained. Even when at that point there is no practical information for a specific field, that is already information in terms of Shannon entropy. From here, there could be a natural representation of the data as particles for Boltzmann entropy. –  labotsirc Sep 28 '12 at 3:40
Changing the constant $K_B$ is the same thing as changing the base of the logarithm, because $\log_a x = \log_b x / \log_b a$. There is no difference between Shannon entropy and Boltzmann entropy except for a change of units. –  Peter Shor Sep 29 '12 at 23:27
+1 for that observation –  labotsirc Sep 30 '12 at 13:44

As has already been answered, Shannon entropy and Boltzman entropy are the same thing, although they are measured in different units. You also asked whether there is a practical link. It may not be practical yet, but the idea of algorithmic cooling uses the link between these two concepts, and has indeed been experimentally demonstrated.

-