Safe Haskell | None |
---|
A gradient is represented by an IntMap from gradient indices to values. Elements with no associated values in the gradient are assumed to have a 0 value assigned. Such elements are not interesting: when adding the gradient to the vector of parameters, only nonzero elements are taken into account.
Each value associated with a gradient position is a pair of positive and negative components. They are stored separately to ensure high accuracy of computation results. Besides, both positive and negative components are stored in a logarithmic domain.
Documentation
type Grad = IntMap LogSignedSource
Gradient with nonzero values stored in a logarithmic domain. Since values equal to zero have no impact on the update phase of the SGD method, it is more efficient to not to store those components in the gradient.
add :: Grad -> Int -> Double -> GradSource
Add normal-domain double to the gradient at the given position.
addL :: Grad -> Int -> LogSigned -> GradSource
Add log-domain, singed number to the gradient at the given position.
fromList :: [(Int, Double)] -> GradSource
Construct gradient from a list of (index, value) pairs. All values from the list are added at respective gradient positions.
fromLogList :: [(Int, LogSigned)] -> GradSource
Construct gradient from a list of (index, signed, log-domain number) pairs. All values from the list are added at respective gradient positions.