Documentation

Gradient of the log posterior function.

Masses of parameters.

NOTE: Full specification of a mass matrix including off-diagonal elements is not supported.

NOTE: Parameters without masses (Nothing) are not changed by the Hamiltonian proposal.

The masses roughly describe how reluctant the particle moves through the state space. If a parameter has higher mass, the momentum in this direction will be changed less by the provided gradient, than when the same parameter has lower mass.

The proposal is more efficient if masses are assigned according to the inverse (co)-variance structure of the posterior function. That is, parameters changing on larger scales should have lower masses than parameters changing on lower scales. In particular, and for a diagonal mass matrix, the optimal masses are the inverted variances of the parameters distributed according to the posterior function.

Of course, the scales of the parameters of the posterior function are usually unknown. Often, it is sufficient to

• set the masses to identical values roughly scaled with the inverted estimated average variance of the posterior function; or even to
• set all masses to 1.0, and trust the tuning algorithm (see HTuneMassesAndLeapfrog) to find the correct values.

Mean leapfrog trajectory length \(L\).

Number of leapfrog steps per proposal.

To avoid problems with ergodicity, the actual number of leapfrog steps is sampled proposal from a discrete uniform distribution over the interval \([\text{floor}(0.8L),\text{ceiling}(1.2L)]\).

For a discussion of ergodicity and reasons why randomization is important, see [1] p. 15; also mentioned in [2] p. 304.

NOTE: To avoid errors, the left bound has an additional hard minimum of 1, and the right bound is required to be larger equal than the left bound.

Usually set to 10, but larger values may be desirable.

Mean of leapfrog scaling factor \(\epsilon\).

Determines the size of each leapfrog step.

To avoid problems with ergodicity, the actual leapfrog scaling factor is sampled per proposal from a continuous uniform distribution over the interval \((0.8\epsilon,1.2\epsilon]\).

For a discussion of ergodicity and reasons why randomization is important, see [1] p. 15; also mentioned in [2] p. 304.

Usually set such that \( L \epsilon = 1.0 \), but smaller values may be required if acceptance rates are low.

Tuning settings.

Tuning of leapfrog parameters:

We expect that the larger the leapfrog step size the larger the proposal step size and the lower the acceptance ratio. Consequently, if the acceptance rate is too low, the leapfrog step size is decreased and vice versa. Further, the leapfrog trajectory length is scaled such that the product of the leapfrog step size and trajectory length stays constant.

Tuning of masses:

The variances of all parameters of the posterior distribution obtained over the last auto tuning interval is calculated and the masses are amended using the old masses and the inverted variances. If, for a specific coordinate, the sample size is too low, or if the calculated variance is out of predefined bounds, the mass of the affected position is not changed.

Specifications for Hamilton Monte Carlo proposal.

Hamiltonian Monte Carlo proposal.

The Applicative and Traversable instances are used for element-wise operations.

Assume a zip-like Applicative instance so that cardinality remains constant.

NOTE: The desired acceptance rate is 0.65, although the dimension of the proposal is high.

NOTE: The speed of this proposal can change drastically when tuned because the leapfrog trajectory length is changed.