The Tower type is a concrete representation of a higher-order Dual number, meaning a number augmented with a tower of derivatives. These generalize the Dual numbers of Clifford (1873), which hold only a first derivative. They can be converted to formal power series via division by the sequence of factorials.

The lift function injects a primal number into the domain of derivative towers, with a zero tower. If generalized dual numbers were a monad, lift would be return.

The primal function finds the primal value from a derivative | tower. The inverse of lift.

The diffUU function calculates the first derivative of a scalar-to-scalar function.

The diffUF function calculates the first derivative of scalar-to-nonscalar function.

The diffMU function calculate the product of the Jacobian of a nonscalar-to-scalar function with a given vector. Aka: directional derivative.

The diffMF function calculates the product of the Jacobian of a nonscalar-to-nonscalar function with a given vector. Aka: directional derivative.

The diff2UU function calculates the value and derivative, as a pair, of a scalar-to-scalar function.

The diffUF2 function calculates the value and derivative, as a pair, of a scalar-to-nonscalar function.

The diffMU2 function calculates the value and directional derivative, as a pair, of a nonscalar-to-scalar function.

The diffMF2 function calculates the value and directional derivative, as a pair, of a nonscalar-to-nonscalar function.

The diffsUU function calculates a list of derivatives of a scalar-to-scalar function. The 0-th element of the list is the primal value, the 1-st element is the first derivative, etc.

The diffsUF function calculates an infinite list of derivatives of a scalar-to-nonscalar function. The 0-th element of the list is the primal value, the 1-st element is the first derivative, etc.

The diffsMU function calculates an infinite list of derivatives of a nonscalar-to-scalar function. The 0-th element of the list is the primal value, the 1-st element is the first derivative, etc. The input is a (possibly truncated) list of the primal, first derivative, etc, of the input.

The diffsMF function calculates an infinite list of derivatives of a nonscalar-to-nonscalar function. The 0-th element of the list is the primal value, the 1-st element is the first derivative, etc. The input is a (possibly truncated) list of the primal, first derivative, etc, of the input.

The diffs0UU function is like diffsUU except the output is zero padded.

The diffs0UF function is like diffsUF except the output is zero padded.

The diffs0MU function is like diffsMU except the output is zero padded.

The diffs0MF function is like diffsMF except the output is zero padded.

The diff function is a synonym for diffUU.

The diff2 function is a synonym for diff2UU.

The diffs function is a synonym for diffsUU.

The diffs0 function is a synonym for diffs0UU.

The grad function calculates the gradient of a nonscalar-to-scalar function, using n invocations of forward AD, where n is the input dimmensionality. NOTE: this is O(n) inefficient as compared to reverse AD.

The jacobian function calcualtes the Jacobian of a nonscalar-to-nonscalar function, using n invocations of forward AD, where n is the input dimmensionality.

The zeroNewton function finds a zero of a scalar function using Newton's method; its output is a stream of increasingly accurate results. (Modulo the usual caveats.)

TEST CASE: take 10 $ zeroNewton (\x->x^2-4) 1 -- converge to 2.0

TEST CASE :module Data.Complex Numeric.FAD take 10 $ zeroNewton ((+1).(^2)) (1 :+ 1) -- converge to (0 :+ 1)

The inverseNewton function inverts a scalar function using Newton's method; its output is a stream of increasingly accurate results. (Modulo the usual caveats.)

TEST CASE: take 10 $ inverseNewton sqrt 1 (sqrt 10) -- converge to 10

The fixedPointNewton function find a fixedpoint of a scalar function using Newton's method; its output is a stream of increasingly accurate results. (Modulo the usual caveats.)

The extremumNewton function finds an extremum of a scalar function using Newton's method; produces a stream of increasingly accurate results. (Modulo the usual caveats.)

The argminNaiveGradient function performs a multivariate optimization, based on the naive-gradient-descent in the file stalingrad/examples/flow-tests/pre-saddle-1a.vlad from the VLAD compiler Stalingrad sources. Its output is a stream of increasingly accurate results. (Modulo the usual caveats.) The gradient is calculated using Forward AD, which is O(n) inefficient as compared to Reverse AD, where n is the input dimensionality.

The primalUU function lowers a function over dual numbers to a function in the primal domain, where the function is scalar-to-scalar.

The primalUF function lowers a function over dual numbers to a function over primals, where the function is scalar-to-nonscalar.

The primalFU function lowers a function over dual numbers to a function over primals where the function is nonscalar-to-scalar.

The primalFF function lowers a function over dual numbers to a function over primals where the function is nonscalar-to-nonscalar.

The taylor function evaluate a Taylor series of the given function around the given point with the given delta. It returns a list of increasingly higher-order approximations.

EXAMPLE: taylor exp 0 1

The taylor2 function evaluates a two-dimensional Taylor series of the given function. This is calculated by nested application of the taylor function, and the exported signature reflects this.