Backprop morphism. Base type for an infinitely differentiable object. It depends on categorical type cat that is mostly common (->), see BackpropFunc which by it's definition is equivalent to

data BackpropFunc input output = forall cache. MkBackpropFunc {
 call     :: input -> output,
 forward  :: BackpropFunc input (output, cache),
 backward :: BackpropFunc (output, cache) input
}

The diagram below illustrates the how it works for the first derivative. Consider the role of function f in the derivative of the composition g(f(h(...))).

  h        ·                  f                   ·        g
           ·                                      ·
           ·               forward                ·
           · --- input  >-----+-----> output >--- ·
           ·                  V                   ·
 ...       ·                  |                   ·       ...
           ·                  | cache             ·
           ·                  |                   ·
           ·                  V                   ·
           · --< dInput <-----+-----< dOutput <-- ·
           ·               backward               ·

Notice that forward and backward are of type BackpropFunc but not (->). This is needed for further differentiation. However for the first derivative this difference can be ignored.

The return type of forward contains additional term cache. It is needed to save and transfer data calculated in the forward step to the backward step for reuse. See an example in

Differentiation with logging section .

Remark

Simple internal category object extraction.

Returns forward category. In the case cat = (->), the method coincides with Backprop cat input output itself but the output contains an additional data term cache with some calculation result that can be reused on in backward.

Returns backward category. In the case cat = (->), the method takes the additional data term cache that is calculated in forward.

Interface for categories cat and value types x that support starting the backpropagation. For example for (->) and Float we are able to start the backpropagation like f(g(x)) -> 1 · f'(g(x)) · ... because f is a Float valued (scalar) function. Calculating Jacobians is not currently implemented.

Morphism that connects forward and backward chain. Usually it is just 1 that is supposed to be multiplied on the derivative of the top function.

Implementation of the process illustrated in the diagram. The first argument is a backprop morphism y -> dy The second argument is a backprop morphism x -> y The output is the backprop x -> dx build according the diagram

Backpropagation derivative in terms of backprop morphisms.

Backpropagation ns derivative in terms of backprop morphisms.

Backpropagation derivative as categorical object. If cat is (->) the output is simply a function.

Examples of usage

>>> import InfBackprop (sin)
>>> import Prelude (Float)
>>> derivative sin (0 :: Float)
1.0

Backpropagation derivative of order n as categorical object. If cat is (->) the output is simply a function.

Examples of usage

>>> import InfBackprop (pow, const)
>>> import Prelude (Float, fmap)
>>> myFunc = (pow 2) :: Backprop (->) Float Float

>>> fmap (derivativeN 0 myFunc) [-3, -2, -1, 0, 1, 2, 3]
[9.0,4.0,1.0,0.0,1.0,4.0,9.0]

>>> fmap (derivativeN 1 myFunc) [-3, -2, -1, 0, 1, 2, 3]
[-6.0,-4.0,-2.0,0.0,2.0,4.0,6.0]

>>> fmap (derivativeN 2 myFunc) [-3, -2, -1, 0, 1, 2, 3]
[2.0,2.0,2.0,2.0,2.0,2.0,2.0]

>>> fmap (derivativeN 3 myFunc) [-3, -2, -1, 0, 1, 2, 3]
[0.0,0.0,0.0,0.0,0.0,0.0,0.0]

Infinitely differentiable function. The definition of the type synonym is equivalent to

data BackpropFunc input output = forall cache. MkBackpropFunc {
   call     :: input -> output,
   forward  :: BackpropFunc input (output, cache),
   backward :: BackpropFunc (output, cache) input
 }

See Backprop for details.

Examples of usage

>>> import Prelude (fmap, Float)
>>> import InfBackprop (pow, call, derivative)
>>> myFunc = pow 2 :: BackpropFunc Float Float
>>> f = call myFunc :: Float -> Float
>>> fmap f [-3, -2, -1, 0, 1, 2, 3]
[9.0,4.0,1.0,0.0,1.0,4.0,9.0]
>>> df = derivative myFunc :: Float -> Float
>>> fmap df [-3, -2, -1, 0, 1, 2, 3]
[-6.0,-4.0,-2.0,0.0,2.0,4.0,6.0]

Infinitely differentiable constant function.

Examples of usage

>>> import Prelude (Float)
>>> import InfBackprop (call, derivative, derivativeN)

>>> call (const 5) ()
5

>>> derivative (const (5 :: Float)) 42
0

>>> derivativeN 2 (const (5 :: Float)) 42
0.0

Lifts a backprop function morphism to the corresponding pure Kleisli morphism.