| Safe Haskell | None |
|---|---|
| Language | Haskell2010 |
Goal.Graphical.Learning
Description
A collection of algorithms for optimizing harmoniums.
Synopsis
- expectationMaximization :: (DuallyFlatExponentialFamily (AffineHarmonium f y x z w), ExponentialFamily z, Map Natural f x y, Bilinear f y x, Translation z y, Translation w x, LegendreExponentialFamily w) => Sample z -> (Natural # AffineHarmonium f y x z w) -> Natural # AffineHarmonium f y x z w
- expectationMaximizationAscent :: (LegendreExponentialFamily (AffineHarmonium f y x z w), ExponentialFamily z, Map Natural f x y, Bilinear f y x, Translation z y, Translation w x, LegendreExponentialFamily w) => Double -> GradientPursuit -> Sample z -> (Natural # AffineHarmonium f y x z w) -> [Natural # AffineHarmonium f y x z w]
- gibbsExpectationMaximization :: (ExponentialFamily z, Map Natural f x y, Manifold w, Map Natural f y x, Translation z y, Translation w x, SamplePoint y ~ SamplePoint z, SamplePoint w ~ SamplePoint x, ExponentialFamily y, Generative Natural w, ExponentialFamily x, Generative Natural z, Manifold (AffineHarmonium f y x z w), Bilinear f y x, LegendreExponentialFamily w) => Double -> Int -> Int -> GradientPursuit -> Sample z -> (Natural # AffineHarmonium f y x z w) -> Chain Random (Natural # AffineHarmonium f y x z w)
- latentProcessExpectationMaximization :: (ConjugatedLikelihood g x x w w, ConjugatedLikelihood f y x z w, Transition Natural Mean w, Transition Natural Mean (AffineHarmonium g x x w w), Transition Mean Natural w, Transition Mean Natural (AffineHarmonium f y x z w), Transition Mean Natural (AffineHarmonium g x x w w), Manifold (AffineHarmonium g x x w w), Bilinear g x x, Map Natural f x y, Bilinear f y x, SamplePoint y ~ SamplePoint z) => Observations (LatentProcess f g y x z w) -> (Natural # LatentProcess f g y x z w) -> Natural # LatentProcess f g y x z w
- latentProcessExpectationMaximizationAscent :: (ConjugatedLikelihood g x x w w, ConjugatedLikelihood f y x z w, DuallyFlatExponentialFamily w, LegendreExponentialFamily (AffineHarmonium f y x z w), LegendreExponentialFamily (AffineHarmonium g x x w w), Bilinear g x x, Map Natural f x y, Bilinear f y x, SamplePoint y ~ SamplePoint z) => Double -> Int -> GradientPursuit -> [Sample z] -> (Natural # LatentProcess f g y x z w) -> Natural # LatentProcess f g y x z w
- harmoniumInformationProjectionDifferential :: (Map Natural f y x, LegendreExponentialFamily z, SamplePoint w ~ SamplePoint x, Translation z y, ExponentialFamily x, ExponentialFamily w, Generative Natural w) => Int -> (Natural # AffineHarmonium f y x z w) -> (Natural # w) -> Random (Mean # w)
- contrastiveDivergence :: (Generative Natural z, ExponentialFamily z, Translation w x, Generative Natural w, ExponentialFamily y, Translation z y, LegendreExponentialFamily w, Bilinear f y x, Map Natural f x y, Map Natural f y x, SamplePoint y ~ SamplePoint z, SamplePoint x ~ SamplePoint w, ExponentialFamily x) => Int -> Sample z -> (Natural # AffineHarmonium f y x z w) -> Random (Mean # AffineHarmonium f y x z w)
Expectation Maximization
expectationMaximization :: (DuallyFlatExponentialFamily (AffineHarmonium f y x z w), ExponentialFamily z, Map Natural f x y, Bilinear f y x, Translation z y, Translation w x, LegendreExponentialFamily w) => Sample z -> (Natural # AffineHarmonium f y x z w) -> Natural # AffineHarmonium f y x z w Source #
A single iteration of EM for Harmonium based models.
expectationMaximizationAscent :: (LegendreExponentialFamily (AffineHarmonium f y x z w), ExponentialFamily z, Map Natural f x y, Bilinear f y x, Translation z y, Translation w x, LegendreExponentialFamily w) => Double -> GradientPursuit -> Sample z -> (Natural # AffineHarmonium f y x z w) -> [Natural # AffineHarmonium f y x z w] Source #
Ascent of the EM objective on harmoniums for when the expectation step can't be computed in closed-form. The convergent harmonium distribution of the output harmonium-list is the result of 1 iteration of the EM algorithm.
gibbsExpectationMaximization Source #
Arguments
| :: (ExponentialFamily z, Map Natural f x y, Manifold w, Map Natural f y x, Translation z y, Translation w x, SamplePoint y ~ SamplePoint z, SamplePoint w ~ SamplePoint x, ExponentialFamily y, Generative Natural w, ExponentialFamily x, Generative Natural z, Manifold (AffineHarmonium f y x z w), Bilinear f y x, LegendreExponentialFamily w) | |
| => Double | |
| -> Int | |
| -> Int | |
| -> GradientPursuit | |
| -> Sample z | Observations |
| -> (Natural # AffineHarmonium f y x z w) | Current Harmonium |
| -> Chain Random (Natural # AffineHarmonium f y x z w) | Harmonium Chain |
Ascent of the EM objective on harmoniums for when the expectation step can't be computed in closed-form. The convergent harmonium distribution of the output harmonium-list is the result of 1 iteration of the EM algorithm.
latentProcessExpectationMaximization :: (ConjugatedLikelihood g x x w w, ConjugatedLikelihood f y x z w, Transition Natural Mean w, Transition Natural Mean (AffineHarmonium g x x w w), Transition Mean Natural w, Transition Mean Natural (AffineHarmonium f y x z w), Transition Mean Natural (AffineHarmonium g x x w w), Manifold (AffineHarmonium g x x w w), Bilinear g x x, Map Natural f x y, Bilinear f y x, SamplePoint y ~ SamplePoint z) => Observations (LatentProcess f g y x z w) -> (Natural # LatentProcess f g y x z w) -> Natural # LatentProcess f g y x z w Source #
Direct expectation maximization for LatentProcesses.
latentProcessExpectationMaximizationAscent :: (ConjugatedLikelihood g x x w w, ConjugatedLikelihood f y x z w, DuallyFlatExponentialFamily w, LegendreExponentialFamily (AffineHarmonium f y x z w), LegendreExponentialFamily (AffineHarmonium g x x w w), Bilinear g x x, Map Natural f x y, Bilinear f y x, SamplePoint y ~ SamplePoint z) => Double -> Int -> GradientPursuit -> [Sample z] -> (Natural # LatentProcess f g y x z w) -> Natural # LatentProcess f g y x z w Source #
Expectation maximization for LatentProcesses approximated through
gradient ascent.
Differentials
harmoniumInformationProjectionDifferential Source #
Arguments
| :: (Map Natural f y x, LegendreExponentialFamily z, SamplePoint w ~ SamplePoint x, Translation z y, ExponentialFamily x, ExponentialFamily w, Generative Natural w) | |
| => Int | |
| -> (Natural # AffineHarmonium f y x z w) | Harmonium |
| -> (Natural # w) | Model Distribution |
| -> Random (Mean # w) | Differential Estimate |
The differential of the dual relative entropy. Minimizing this results in the information projection of the model against the marginal distribution of the given harmonium. This is more efficient than the generic version.
contrastiveDivergence Source #
Arguments
| :: (Generative Natural z, ExponentialFamily z, Translation w x, Generative Natural w, ExponentialFamily y, Translation z y, LegendreExponentialFamily w, Bilinear f y x, Map Natural f x y, Map Natural f y x, SamplePoint y ~ SamplePoint z, SamplePoint x ~ SamplePoint w, ExponentialFamily x) | |
| => Int | The number of contrastive divergence steps |
| -> Sample z | The initial states of the Gibbs chains |
| -> (Natural # AffineHarmonium f y x z w) | The harmonium |
| -> Random (Mean # AffineHarmonium f y x z w) | The gradient estimate |
Contrastive divergence on harmoniums (Hinton, 2019).