4
$\begingroup$

I have two random variables $X$ and $Y$. $X$ follows Poisson-Binomial distribution with parameters $\{q_1, \ldots, q_k\}$. Thus, $X$ can take values in the set $\{0,1,\ldots,k\}$.

$Y$ is a binary random variable. $Y=0$ with probability $\frac{0.1}{1+i}$ when $X=i$.

I am interested in the mutual information between $X$ and $Y$ denoted by $I(X;Y)$.

Specifically, I have a conjecture that the mutual information $I(X;Y)$ is maximum when all $q_i$'s are equal which leads to a Binomial distribution for $X$.

Can someone help me prove this? Or provide a counter example?

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Do you have an intuition or justification for this conjecture? $\endgroup$
    – Clement C.
    Feb 9, 2022 at 5:48
  • $\begingroup$ @ClementC. Yes, First of all, Binomial distribution has the maximum entropy among Poisson- binomial distributions. Also, for a given mean, the variance is maximum when all probability parameters are equal and we have a binomial distribution. So, I am hoping somehow the symmetry can be used to claim the conjecture. ie, From a Poisson-Binomial, you can always create a BInomial with a better mutual information by mixing the sampling probabilities. I don't know how to establish this though. $\endgroup$
    – wanderer
    Feb 9, 2022 at 16:50
  • 1
    $\begingroup$ There is a significant probability I messed up the "coding" in Mathematica, but, if not, it seems that a version of the conjecture is false (for a given value of $q_1$, it's maximised when all equal) in the case $k=2$: wolframcloud.com/obj/ccanonne/Published/conjecture-mi-pbd.nb (see the two plots at the end) $\endgroup$
    – Clement C.
    Feb 11, 2022 at 4:12
  • 1
    $\begingroup$ yes, this is what I laboriously tried to say above. What is not true is that, fixing an arbitrary $q_1$, the maximum is for $q_2=q_1$. What might be true (supported by this) is that, optimizing over both $q_1,q_2$, the max is realized for some pair where they are equal. $\endgroup$
    – Clement C.
    Feb 11, 2022 at 5:09
  • 1
    $\begingroup$ @ClementC, Yes, I am interested in joint optimization over all $q_i$'s. This also is in accordance with the intuitions I stated in my first comment. They are all focused on for a given mean $(\sum q_i)$. So, if someone can prove that for a given mean, my conjecture holds. The extension is trivial. $\endgroup$
    – wanderer
    Feb 11, 2022 at 5:18

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.