Bernstein–von Mises theorem

In Bayesian inference, the Bernstein–von Mises theorem provides the basis for using Bayesian credible sets for confidence statements in parametric models. It states that under some conditions, a posterior distribution converges in total variation distance to a multivariate normal distribution centered at the maximum likelihood estimator ${\displaystyle {\widehat {\theta }}_{n}}$ with covariance matrix given by ${\displaystyle n^{-1}{\mathcal {I}}(\theta _{0})^{-1}}$, where ${\displaystyle \theta _{0}}$ is the true population parameter and ${\displaystyle {\mathcal {I}}(\theta _{0})}$ is the Fisher information matrix at the true population parameter value:

${\displaystyle ||P(\theta |x_{1},\dots x_{n})-{\mathcal {N}}({\widehat {\theta }}_{n},n^{-1}{\mathcal {I}}(\theta _{0})^{-1})||_{\mathrm {TV} }\xrightarrow {P_{\theta _{0}}} =0}$

The Bernstein–von Mises theorem links Bayesian inference with frequentist inference. It assumes there is some true probabilistic process that generates the observations, as in frequentism, and then studies the quality of Bayesian methods of recovering that process, and making uncertainty statements about that process. In particular, it states that asymptotically, many Bayesian credible sets of a certain credibility level ${\displaystyle \alpha }$ will act as confidence sets of confidence level ${\displaystyle \alpha }$, which allows for the interpretation of Bayesian credible sets.