I am trying to think through the number of samples that I would need to estimate the parameters of a Boltzmann partition function to a desirable precision.

Suppose that there are N possible states of the world, with the probability of state i being observed equal to

$$Pr(i | \theta) = \frac{e^{-\theta_i}}{ \sum_{i=1}^N e^{-\theta_i}}.$$

I don't know what the values of $\{\theta_i\}_{i=1}^N$ are, but I can sample independent observations from the set {1,...,N} of possible states of the world, distributed according to the distribution $Pr(i)$ given above.

Let's say I draw $m$ observations $X_1,...,X_m \in \{1,...,N\}$, and define the maximum likelihood estimator $$\widehat{\theta} = \arg \max_{\theta'} \sum_{j=1}^m log Pr(X_j | \theta)$$

When $m \to \infty$, then $\widehat{\theta} = \theta$. However, I'm not sure how many samples $m$ I would need to ensure that the estimate $\widehat{\theta}$ is probably approximately correct. That is, how large does $m$ need to be in terms of $\epsilon,N,\delta$ to ensure that $Pr ( \| \widehat{\theta} - \theta \|_{\infty} > \epsilon ) < \delta$?

up vote 1 down vote accepted

Under appropriate smoothness conditions (satisfied by the Boltzmann distribution), the maximum likelihood is asymptotically normal. Meaning: the vector $\sqrt n(\theta-\hat\theta)$ will have approximately normal distribution, with mean $0$ and covariance matrix given by the inverse Fisher matrix, see Section 9 here: http://www.stat.cmu.edu/~larry/=stat705/Lecture9.pdf

  • I initially wanted to compute the Fisher matrix at $\theta$ but this seems to be non-trivial in this case -- I'd be curious if anyone can do it in closed form. – Aryeh Oct 10 at 21:17

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.