For $p\in[0,1]^{\mathbb{N}}$ and $\alpha\ge1$, define $$ H_\alpha(p) = \sum_{i\in\mathbb{N}}p_i|\log(p_i)|^\alpha. $$ When $\sum_i p_i=1$ and $\alpha=1$, $H_1(p)$ is just the Shannon entropy of the distribution $p$. Has anyone encountered the object $H_\alpha$ anywhere in the literature? A reference would be much appreciated. I'm thinking of calling $H_\alpha$ hyperentropy for $\alpha>1$; is that term already taken by chance?
$\begingroup$
$\endgroup$
2
-
2$\begingroup$ Not an answer, but linking math.stackexchange.com/questions/3781318/… $\endgroup$– Clement C.Commented Sep 21, 2020 at 23:19
-
$\begingroup$ Thanks!! I'm curious about the other guy's motivation... $\endgroup$– AryehCommented Sep 21, 2020 at 23:20
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
It's the $\alpha^{\mathrm{th}}$ moment of the Tribus surprisal.
This generalizes the statement that entropy = expected surprisal.
Or in Ross's textbook, "expected surprise".
answered Sep 23, 2020 at 19:42
-
2$\begingroup$ So... "moment of surprise"? $\endgroup$ Commented Sep 23, 2020 at 19:49
-
$\begingroup$ I searched Ross's book for "expected surprise" and nothing came up -- what page is it on? $\endgroup$– AryehCommented Sep 23, 2020 at 22:33
-
$\begingroup$ Any chance you have Trubus's book in pdf? $\endgroup$– AryehCommented Sep 23, 2020 at 22:38
-
1$\begingroup$ You can search for "surprise" $\endgroup$ Commented Sep 24, 2020 at 2:39
$\begingroup$
$\endgroup$
So we ended up calling this quantity the $\alpha$th moment of information and proving some inequalities about it: https://arxiv.org/abs/2004.12680 (paper to appear in the NIPS 2021 conference).