Jeffreys prior

In Bayesian statistics, the Jeffreys prior is a non-informative prior distribution for a parameter space. Named after Sir Harold Jeffreys, its density function is proportional to the square root of the determinant of the Fisher information matrix:

$${\displaystyle p\left(\theta \right)\propto \left|I(\theta )\right|^{1/2}.\,}$$

It has the key feature that it is invariant under a change of coordinates for the parameter vector ${\textstyle \theta }$. That is, the relative probability assigned to a volume of a probability space using a Jeffreys prior will be the same regardless of the parameterization used to define the Jeffreys prior. This makes it of special interest for use with scale parameters. As a concrete example, a Bernoulli distribution can be parameterized by the probability of occurrence p, or by the odds r = p / (1 − p). A uniform prior on one of these is not the same as a uniform prior on the other, even accounting for reparameterization in the usual way, but the Jeffreys prior on one reparameterizes to the Jeffreys prior on the other.

In maximum likelihood estimation of exponential family models, penalty terms based on the Jeffreys prior were shown to reduce asymptotic bias in point estimates.