3
$\begingroup$

For $p\in[0,1]^{\mathbb{N}}$ and $\alpha\ge1$, define $$ H_\alpha(p) = \sum_{i\in\mathbb{N}}p_i|\log(p_i)|^\alpha. $$ When $\sum_i p_i=1$ and $\alpha=1$, $H_1(p)$ is just the Shannon entropy of the distribution $p$. Has anyone encountered the object $H_\alpha$ anywhere in the literature? A reference would be much appreciated. I'm thinking of calling $H_\alpha$ hyperentropy for $\alpha>1$; is that term already taken by chance?

$\endgroup$
2

2 Answers 2

2
$\begingroup$

It's the $\alpha^{\mathrm{th}}$ moment of the Tribus surprisal.

This generalizes the statement that entropy = expected surprisal.

Or in Ross's textbook, "expected surprise".

$\endgroup$
4
  • 2
    $\begingroup$ So... "moment of surprise"? $\endgroup$ Sep 23, 2020 at 19:49
  • $\begingroup$ I searched Ross's book for "expected surprise" and nothing came up -- what page is it on? $\endgroup$
    – Aryeh
    Sep 23, 2020 at 22:33
  • $\begingroup$ Any chance you have Trubus's book in pdf? $\endgroup$
    – Aryeh
    Sep 23, 2020 at 22:38
  • 1
    $\begingroup$ You can search for "surprise" $\endgroup$ Sep 24, 2020 at 2:39
1
$\begingroup$

So we ended up calling this quantity the $\alpha$th moment of information and proving some inequalities about it: https://arxiv.org/abs/2004.12680 (paper to appear in the NIPS 2021 conference).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.