Define $[n] = \{1, 2, ..., n\}$. Given a distribution $P : \{0, 1\}^{[n]} \rightarrow [0, 1]$ and a subset $S \subseteq [n]$, we can define the $S$-marginal of $P$, $P_S : \{0, 1\}^S \rightarrow [0, 1]$ as $$P_S(x) = \sum_{y \in \{0, 1\}^{[n] - S}} P(x \cup y)$$

Given a set of sets $\Sigma \subseteq 2^{[n]}$ indexing probability distributions $\phi_S : \{0, 1\}^S \rightarrow [0, 1]$, we can define $$P = argmax_{P : \forall S \in \Sigma, P_S = \phi_S, \sum_{x \in \{0, 1\}^{[n]}} P(x) = 1} H(P)$$ where $H(P)$ is the Shannon entropy of $P$.

In other words, we can define $P$ as the maximum entropy distribution with the marginals $\{ \phi_S \mid S \in \Sigma \}$. Define $support(P) = \{ x \mid x \in \{0, 1\}^{[n]}, P(x) > 0 \}$.

You'll notice each marginal constraints the support to a set:

$$ supp_n(S) = \{ x \cup y \mid x \in \{0, 1\}^S, \phi_S(x) > 0, y \in \{0, 1\}^{[n] - S}\} $$

and its clear that $support(P) \subseteq \bigcap_{S \in \Sigma} supp_n(S)$.

My question is if $support(P) = \bigcap_{S \in \Sigma} supp_n(S)$ when $P$ is well-defined. My use case for this has to do with analyzing a particular model for distributions which compares in interesting ways to circuit-based models for distributions.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.