3
$\begingroup$

For $p\in[0,1]^{\mathbb{N}}$ and $\alpha\ge1$, define $$ H_\alpha(p) = \sum_{i\in\mathbb{N}}p_i|\log(p_i)|^\alpha. $$ When $\sum_i p_i=1$ and $\alpha=1$, $H_1(p)$ is just the Shannon entropy of the distribution $p$. Has anyone encountered the object $H_\alpha$ anywhere in the literature? A reference would be much appreciated. I'm thinking of calling $H_\alpha$ hyperentropy for $\alpha>1$; is that term already taken by chance?

$\endgroup$
2
2
$\begingroup$

It's the $\alpha^{\mathrm{th}}$ moment of the Tribus surprisal.

This generalizes the statement that entropy = expected surprisal.

Or in Ross's textbook, "expected surprise".

$\endgroup$
4
  • 2
    $\begingroup$ So... "moment of surprise"? $\endgroup$ Sep 23 '20 at 19:49
  • $\begingroup$ I searched Ross's book for "expected surprise" and nothing came up -- what page is it on? $\endgroup$
    – Aryeh
    Sep 23 '20 at 22:33
  • $\begingroup$ Any chance you have Trubus's book in pdf? $\endgroup$
    – Aryeh
    Sep 23 '20 at 22:38
  • 1
    $\begingroup$ You can search for "surprise" $\endgroup$ Sep 24 '20 at 2:39
1
$\begingroup$

So we ended up calling this quantity the $\alpha$th moment of information and proving some inequalities about it: https://arxiv.org/abs/2004.12680 (paper to appear in the NIPS 2021 conference).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.