1
$\begingroup$

Suppose $X$ is a message which takes values on the set $\{x_1, \dots, x_m\}$ with probability distribution $P_X$. We transmit the message $X$ over the channel $P_{Y|X}$ which outputs $Y$ taking values on finite alphabet $\mathcal{Y}$. Suppose we take $n$ independent samples from channel output; namely, $Y^n=(Y_1, Y_2, \dots, Y_n)$ with distribution $\prod_{i=1}^nP_{Y|X}(y_k|x)$ where $x$ is the message sent over the channel.

Hence, we have $m$ distributions $\{P_{Y^n|X}(\cdot|x_1), \dots, P_{Y^n|X}(\cdot|x_m)\}$. I am looking for a deterministic randomness extractor which works for all of these $m$ distributions, that is, a randomness $Ext:\mathcal{Y^n}$ such that for given $Y^n$ distributed as one of these $m$ distribution $Ext(Y^n)$ is $\epsilon-$close to uniform distribution over $\{0,1\}^r$ (in total variation distance sense).

What is the largest random bits that we can extract, i.e., the largest $r$? Is that $\min_{x}H_{\infty}(P_{Y^n|X}(\cdot|x))$?

The motivation of this comes from privacy.

Improve this question
$\endgroup$
8
  • $\begingroup$ What if $Y=x$ deterministically? Then there is no entropy in $Y^m$. I don't see how your assumptions give anything beyond the usual problem of extraction. You want to deterministically extract from m iid samples from one of m distributions. $\endgroup$
    – Thomas
    Apr 24, 2014 at 3:50
  • $\begingroup$ @Thomas, can you plz specify which assumption you mean? yes this is so similar to the usual extraction, I just wanted to make sure that if instead of one distribution, we want to extract pure random bits from a class of distribution, then we can extract at best the minimum over the class of distribution of many random bits, i.e., minimum over $m$ distributions of min-entropy. Is that true? $\endgroup$
    – SAmath
    Apr 24, 2014 at 15:54
  • $\begingroup$ Yes, because an extractor could in particular be applied to any particular member of that class. $\,$ However, you won't necessarily be able to achieve that upper bound. $\;\;\;$ $\endgroup$
    – user6973
    Apr 24, 2014 at 17:01
  • $\begingroup$ @RickyDemer thanks, but i cant get why we cant achieve this. Suppose we apply the extractor to the the worst distribution which yields the shortest random bits, why we can not achieve this when we apply that extractor to any other distribution $\endgroup$
    – SAmath
    Apr 24, 2014 at 20:35
  • $\begingroup$ There is not necessarily a worst distribution. $\:$ The uniform distributions on the sets {"00","10"},{"00","11"},{"00","01"} each have one bit of entropy and there is no deterministic extractor that works on all three of them. $\;\;\;\;$ $\endgroup$
    – user6973
    Apr 24, 2014 at 21:21

1 Answer 1

Reset to default
1
$\begingroup$

I came across this late. Don't know if this question still matters. I am posting this as an answer since it is too long for a comment.

If n can be 1 but m is not too large (say, at most a small exponential in the min-entropy), a random function will be a good extractor with high probability. It will cost you a factor of something like log m in the min-entropy.

However, for large values of n, you should be able to find a good deterministic extractor no matter how large m is since, once you condition on x, the distribution on outputs is IID. In particular, it is symmetric, so you can use as idea like von Neumann's trick or its generalizations to extract a small number of random bits just from the ordering of the elements. That might be enough to get the seed for a strong randomness extractor which could extract the rest of the randomness from the source. You have to be careful, but the idea would be to rely heavily on the IID structure to get a a deterministic extractor. Don't know if that's good enough for the application.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.