0
$\begingroup$

I am reading this classic paper by Claude E. Shannon and I think there may be a couple of errors in his description of the properties of Entropy/Uncertainty. The screenshot shown at the bottom of this post is from page 9. Firstly, I believe there is a typo at the first red circle which should read $10^{50}$. Secondly, I think the equation shown for the value of the entropy $H$ of this problem is also incorrect. I believe the correct equation for $H$ is derived as follows:

events $=\{p_1, p_2, ... , p_{10^{30}}, q_1, q_2, ..., q_{10^{50}}\}$

where events $p_i$ each have a probability $\frac{1}{2}(10^{-30})$ of occurring, and events $q_j$ each have a probability $\frac{1}{2}(10^{-50})$ of occuring

\begin{align} H &= -\sum_{i=1}^{10^{30}} p_i\log{p_i} - \sum_{j=1}^{10^{50}} q_j\log{q_j} \\ &= -\sum_{i=1}^{10^{30}} \frac{1}{2}(10^{-30})\log{(\frac{1}{2}(10^{-30}))} -\sum_{j=1}^{10^{50}} \frac{1}{2}(10^{-50})\log{(\frac{1}{2}(10^{-50}))} \\ &= -\frac{1}{2}\log{(\frac{1}{2}(10^{-30}))} -\frac{1}{2}\log{(\frac{1}{2}(10^{-50}))} \\ &\approx 40 \end{align}

Hence, I think the equation for $H$ should read:

$H = -\frac{1}{2}\log{\frac{1}{2}10^{-30}} -\frac{1}{2}\log{\frac{1}{2}10^{-50}}$

I would really appreciate if someone could let me know if I'm correct or where I'm wrong. I am not doubting Claude, just wondering if the document itself has typos. Thanks!

page 9 part 8 of paper

Improve this question
$\endgroup$
2
  • $\begingroup$ Yes, you are right, these are typos. Btw, you can simplify $-\frac12\log(\frac1210^{-30})-\frac12\log(\frac1210^{-50})$ to $\frac12\log(2\cdot10^{30})+\frac12\log(2\cdot10^{50})=40+\log2$ (I assume from the context these are base-10 logarithms). $\endgroup$
    – Emil Jeřábek
    Mar 22, 2021 at 9:57
  • $\begingroup$ Thanks, yes they're base-10. That's a cool simplification. $\endgroup$
    – nuggimane
    Mar 22, 2021 at 11:01

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.