| Safe Haskell | None |
|---|---|
| Language | Haskell2010 |
Goal.Probability.ExponentialFamily
Description
Definitions for working with exponential families.
Synopsis
- class Statistical x => ExponentialFamily x where
- sufficientStatistic :: SamplePoint x -> Mean # x
- averageSufficientStatistic :: Sample x -> Mean # x
- logBaseMeasure :: Proxy x -> SamplePoint x -> Double
- type LegendreExponentialFamily x = (PotentialCoordinates x ~ Natural, Legendre x, ExponentialFamily x, Transition (PotentialCoordinates x) (Dual (PotentialCoordinates x)) x)
- type DuallyFlatExponentialFamily x = (LegendreExponentialFamily x, DuallyFlat x, Transition (Dual (PotentialCoordinates x)) (PotentialCoordinates x) x)
- exponentialFamilyLogDensities :: (ExponentialFamily x, Legendre x, PotentialCoordinates x ~ Natural) => (Natural # x) -> Sample x -> [Double]
- unnormalizedLogDensities :: forall x. ExponentialFamily x => (Natural # x) -> Sample x -> [Double]
- data Natural
- data Mean
- data Source
- toNatural :: Transition c Natural x => (c # x) -> Natural # x
- toMean :: Transition c Mean x => (c # x) -> Mean # x
- toSource :: Transition c Source x => (c # x) -> Source # x
- relativeEntropy :: DuallyFlatExponentialFamily x => (Mean # x) -> (Natural # x) -> Double
- crossEntropy :: DuallyFlatExponentialFamily x => (Mean # x) -> (Natural # x) -> Double
- relativeEntropyDifferential :: LegendreExponentialFamily x => (Mean # x) -> (Natural # x) -> Mean # x
- stochasticRelativeEntropyDifferential :: ExponentialFamily x => Sample x -> Sample x -> Mean # x
- stochasticInformationProjectionDifferential :: ExponentialFamily x => (Natural # x) -> Sample x -> (SamplePoint x -> Double) -> Mean # x
- exponentialFamilyLogLikelihood :: forall x. LegendreExponentialFamily x => Sample x -> (Natural # x) -> Double
- exponentialFamilyLogLikelihoodDifferential :: LegendreExponentialFamily x => Sample x -> (Natural # x) -> Mean # x
Exponential Families
class Statistical x => ExponentialFamily x where Source #
An ExponentialFamily is a Statistical Manifold \( \mathcal M \)
determined by a fixed-length sufficientStatistic \(s_i\) and a
logBaseMeasure \(\mu\). Each distribution \(P \in \mathcal M\) may then be
identified with Natural parameters \(\theta_i\) such that
\(p(x) \propto e^{\sum_{i=1}^n \theta_i s_i(x)}\mu(x)\). ExponentialFamily
distributions theoretically have a Riemannian geometry, with metric
Tensor given by the Fisher information metric. However, not all
distributions (e.g. the von Mises distribution) afford closed-form
expressions for all the relevant structures.
Minimal complete definition
Methods
sufficientStatistic :: SamplePoint x -> Mean # x Source #
averageSufficientStatistic :: Sample x -> Mean # x Source #
logBaseMeasure :: Proxy x -> SamplePoint x -> Double Source #
Instances
type LegendreExponentialFamily x = (PotentialCoordinates x ~ Natural, Legendre x, ExponentialFamily x, Transition (PotentialCoordinates x) (Dual (PotentialCoordinates x)) x) Source #
When the log-partition function and its derivative of the given
ExponentialFamily may be computed in closed-form, then we refer to it as a
LegendreExponentialFamily.
Note that the log-partition function is the potential of the Legendre
class, and its derivative maps Natural coordinates to Mean coordinates.
type DuallyFlatExponentialFamily x = (LegendreExponentialFamily x, DuallyFlat x, Transition (Dual (PotentialCoordinates x)) (PotentialCoordinates x) x) Source #
When additionally, the (negative) entropy and its derivative of the given
ExponentialFamily may be computed in closed-form, then we refer to it as a
DuallyFlatExponentialFamily.
Note that the negative entropy is the dualPotential of the DuallyFlat class,
and its derivative maps Mean coordinates to Natural coordinates.
exponentialFamilyLogDensities :: (ExponentialFamily x, Legendre x, PotentialCoordinates x ~ Natural) => (Natural # x) -> Sample x -> [Double] Source #
The density of an exponential family distribution that has an exact expression for the log-partition function.
unnormalizedLogDensities :: forall x. ExponentialFamily x => (Natural # x) -> Sample x -> [Double] Source #
The unnormalized log-density of an arbitrary exponential family distribution.
Coordinate Systems
A parameterization in terms of the natural parameters of an exponential family.
Instances
A parameterization in terms of the mean sufficientStatistic of an exponential family.
Instances
A parameterization which represents the standard or typical parameterization of the given manifold, e.g. the Poisson rate or Normal mean and standard deviation.
Instances
Coordinate Transforms
toNatural :: Transition c Natural x => (c # x) -> Natural # x Source #
Expresses an exponential family distribution in Natural coordinates.
toMean :: Transition c Mean x => (c # x) -> Mean # x Source #
Expresses an exponential family distribution in Mean coordinates.
toSource :: Transition c Source x => (c # x) -> Source # x Source #
Expresses an exponential family distribution in Source coordinates.
Entropies
relativeEntropy :: DuallyFlatExponentialFamily x => (Mean # x) -> (Natural # x) -> Double Source #
The relative entropy \(D(P \parallel Q)\), also known as the KL-divergence.
This is simply the canonicalDivergence with its arguments flipped.
crossEntropy :: DuallyFlatExponentialFamily x => (Mean # x) -> (Natural # x) -> Double Source #
A function for computing the cross-entropy, which is the relative entropy plus the entropy of the first distribution.
Differentials
relativeEntropyDifferential :: LegendreExponentialFamily x => (Mean # x) -> (Natural # x) -> Mean # x Source #
The differential of the relative entropy with respect to the Natural parameters of
the second argument.
stochasticRelativeEntropyDifferential Source #
Arguments
| :: ExponentialFamily x | |
| => Sample x | True Samples |
| -> Sample x | Model Samples |
| -> Mean # x | Differential Estimate |
Monte Carlo estimate of the differential of the relative entropy with
respect to the Natural parameters of the second argument, based on samples from
the two distributions.
stochasticInformationProjectionDifferential Source #
Arguments
| :: ExponentialFamily x | |
| => (Natural # x) | Model Distribution |
| -> Sample x | Model Samples |
| -> (SamplePoint x -> Double) | Unnormalized log-density of target distribution |
| -> Mean # x | Differential Estimate |
Maximum Likelihood Instances
exponentialFamilyLogLikelihood :: forall x. LegendreExponentialFamily x => Sample x -> (Natural # x) -> Double Source #
exponentialFamilyLogLikelihoodDifferential :: LegendreExponentialFamily x => Sample x -> (Natural # x) -> Mean # x Source #