I have two random variables $X$ and $Y$. $X$ follows Poisson-Binomial distribution with parameters $\{q_1, \ldots, q_k\}$. Thus, $X$ can take values in the set $\{0,1,\ldots,k\}$.

$Y$ is a binary random variable. $Y=0$ with probability $\frac{0.1}{1+i}$ when $X=i$.

I am interested in the mutual information between $X$ and $Y$ denoted by $I(X;Y)$.

Specifically, I have a conjecture that the mutual information $I(X;Y)$ is maximum when all $q_i$'s are equal which leads to a Binomial distribution for $X$.

Can someone help me prove this? Or provide a counter example?

  • 1
    $\begingroup$ Do you have an intuition or justification for this conjecture? $\endgroup$
    – Clement C.
    Feb 9 at 5:48
  • $\begingroup$ @ClementC. Yes, First of all, Binomial distribution has the maximum entropy among Poisson- binomial distributions. Also, for a given mean, the variance is maximum when all probability parameters are equal and we have a binomial distribution. So, I am hoping somehow the symmetry can be used to claim the conjecture. ie, From a Poisson-Binomial, you can always create a BInomial with a better mutual information by mixing the sampling probabilities. I don't know how to establish this though. $\endgroup$
    – wanderer
    Feb 9 at 16:50
  • 1
    $\begingroup$ There is a significant probability I messed up the "coding" in Mathematica, but, if not, it seems that a version of the conjecture is false (for a given value of $q_1$, it's maximised when all equal) in the case $k=2$: wolframcloud.com/obj/ccanonne/Published/conjecture-mi-pbd.nb (see the two plots at the end) $\endgroup$
    – Clement C.
    Feb 11 at 4:12
  • 1
    $\begingroup$ yes, this is what I laboriously tried to say above. What is not true is that, fixing an arbitrary $q_1$, the maximum is for $q_2=q_1$. What might be true (supported by this) is that, optimizing over both $q_1,q_2$, the max is realized for some pair where they are equal. $\endgroup$
    – Clement C.
    Feb 11 at 5:09
  • 1
    $\begingroup$ @ClementC, Yes, I am interested in joint optimization over all $q_i$'s. This also is in accordance with the intuitions I stated in my first comment. They are all focused on for a given mean $(\sum q_i)$. So, if someone can prove that for a given mean, my conjecture holds. The extension is trivial. $\endgroup$
    – wanderer
    Feb 11 at 5:18


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.