2
$\begingroup$

I am trying to think through the number of samples that I would need to estimate the parameters of a Boltzmann partition function to a desirable precision.

Suppose that there are $N$ possible states of the world, with the probability of state $i$ being observed equal to

$$Pr(i | \theta) = \frac{e^{-\theta_i}}{ \sum_{i=1}^N e^{-\theta_i}}.$$

I don't know what the values of $\{\theta_i\}_{i=1}^N$ are, but I can sample independent observations from the set $\{1,\dots,N\}$ of possible states of the world, distributed according to the distribution $Pr(i)$ given above.

Let's say I draw $m$ observations $X_1,...,X_m \in \{1,...,N\}$, and define the maximum likelihood estimator $$\widehat{\theta} = \arg \max_{\theta'} \sum_{j=1}^m \log Pr(X_j | \theta)$$

When $m \to \infty$, then $\widehat{\theta} = \theta$. However, I'm not sure how many samples $m$ I would need to ensure that the estimate $\widehat{\theta}$ is probably approximately correct. That is, how large does $m$ need to be in terms of $\epsilon,N,\delta$ to ensure that $Pr ( \| \widehat{\theta} - \theta \|_{\infty} > \epsilon ) < \delta$?

$\endgroup$

1 Answer 1

2
$\begingroup$

Under appropriate smoothness conditions (satisfied by the Boltzmann distribution), the maximum likelihood is asymptotically normal. Meaning: the vector $\sqrt n(\theta-\hat\theta)$ will have approximately normal distribution, with mean $0$ and covariance matrix given by the inverse Fisher matrix, see Section 9 here: http://www.stat.cmu.edu/~larry/=stat705/Lecture9.pdf

$\endgroup$
1
  • $\begingroup$ I initially wanted to compute the Fisher matrix at $\theta$ but this seems to be non-trivial in this case -- I'd be curious if anyone can do it in closed form. $\endgroup$
    – Aryeh
    Oct 10, 2018 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.