A stream is a succession of Steps. A Yield produces a single value and the next state of the stream. Stop indicates there are no more values in the stream.

A stream consists of a step function that generates the next step given a current state, and the current state.

A newtype wrapper for the Stream type with a cross product style monad instance.

A Monad bind behaves like a for loop:

>>> :{
Stream.fold Fold.toList $ Stream.unCross $ do
    x <- Stream.mkCross $ Stream.fromList [1,2]
    -- Perform the following actions for each x in the stream
    return x
:}
[1,2]

Nested monad binds behave like nested for loops:

>>> :{
Stream.fold Fold.toList $ Stream.unCross $ do
    x <- Stream.mkCross $ Stream.fromList [1,2]
    y <- Stream.mkCross $ Stream.fromList [3,4]
    -- Perform the following actions for each x, for each y
    return (x, y)
:}
[(1,3),(1,4),(2,3),(2,4)]

Convert a CPS encoded StreamK to direct style step encoded StreamD

Convert a direct style step encoded StreamD to a CPS encoded StreamK

Convert an Unfold into a stream by supplying it an input seed.

>>> s = Stream.unfold Unfold.replicateM (3, putStrLn "hello")
>>> Stream.fold Fold.drain s
hello
hello
hello

A stream that terminates without producing any output, but produces a side effect.

>>> Stream.fold Fold.toList (Stream.nilM (print "nil"))
"nil"
[]

Pre-release

Like cons but fuses an effect instead of a pure value.

Create a singleton stream from a pure value.

>>> fromPure a = a `Stream.cons` Stream.nil
>>> fromPure = pure
>>> fromPure = Stream.fromEffect . pure

Create a singleton stream from a monadic action.

>>> fromEffect m = m `Stream.consM` Stream.nil
>>> fromEffect = Stream.sequence . Stream.fromPure

>>> Stream.fold Fold.drain $ Stream.fromEffect (putStrLn "hello")
hello

Construct a stream from a list of pure values.

Decompose a stream into its head and tail. If the stream is empty, returns Nothing. If the stream is non-empty, returns Just (a, ma), where a is the head of the stream and ma its tail.

Properties:

>>> Nothing <- Stream.uncons Stream.nil
>>> Just ("a", t) <- Stream.uncons (Stream.cons "a" Stream.nil)

This can be used to consume the stream in an imperative manner one element at a time, as it just breaks down the stream into individual elements and we can loop over them as we deem fit. For example, this can be used to convert a streamly stream into other stream types.

All the folds in this module can be expressed in terms of uncons, however, this is generally less efficient than specific folds because it takes apart the stream one element at a time, therefore, does not take adavantage of stream fusion.

foldBreak is a more general way of consuming a stream piecemeal.

>>> :{
uncons xs = do
    r <- Stream.foldBreak Fold.one xs
    return $ case r of
        (Nothing, _) -> Nothing
        (Just h, t) -> Just (h, t)
:}

Fold a stream using the supplied left Fold and reducing the resulting expression strictly at each step. The behavior is similar to foldl'. A Fold can terminate early without consuming the full stream. See the documentation of individual Folds for termination behavior.

Definitions:

>>> fold f = fmap fst . Stream.foldBreak f
>>> fold f = Stream.parse (Parser.fromFold f)

Example:

>>> Stream.fold Fold.sum (Stream.enumerateFromTo 1 100)
5050

Like fold but also returns the remaining stream. The resulting stream would be nil if the stream finished before the fold.

Append a stream to a fold lazily to build an accumulator incrementally.

Example, to continue folding a list of streams on the same sum fold:

>>> streams = [Stream.fromList [1..5], Stream.fromList [6..10]]
>>> f = Prelude.foldl Stream.foldAddLazy Fold.sum streams
>>> Stream.fold f Stream.nil
55

>>> foldAdd = flip Fold.addStream

Fold resulting in either breaking the stream or continuation of the fold. Instead of supplying the input stream in one go we can run the fold multiple times, each time supplying the next segment of the input stream. If the fold has not yet finished it returns a fold that can be run again otherwise it returns the fold result and the residual stream.

Internal

Right associative/lazy pull fold. foldrM build final stream constructs an output structure using the step function build. build is invoked with the next input element and the remaining (lazy) tail of the output structure. It builds a lazy output expression using the two. When the "tail structure" in the output expression is evaluated it calls build again thus lazily consuming the input stream until either the output expression built by build is free of the "tail" or the input is exhausted in which case final is used as the terminating case for the output structure. For more details see the description in the previous section.

Example, determine if any element is odd in a stream:

>>> s = Stream.fromList (2:4:5:undefined)
>>> step x xs = if odd x then return True else xs
>>> Stream.foldrM step (return False) s
True

Right fold, lazy for lazy monads and pure streams, and strict for strict monads.

Please avoid using this routine in strict monads like IO unless you need a strict right fold. This is provided only for use in lazy monads (e.g. Identity) or pure streams. Note that with this signature it is not possible to implement a lazy foldr when the monad m is strict. In that case it would be strict in its accumulator and therefore would necessarily consume all its input.

>>> foldr f z = Stream.foldrM (\a b -> f a <$> b) (return z)

Note: This is similar to Fold.foldr' (the right fold via left fold), but could be more efficient.

Definitions:

>>> drain = Stream.fold Fold.drain
>>> drain = Stream.foldrM (\_ xs -> xs) (return ())

Run a stream, discarding the results.

Definitions:

>>> toList = Stream.foldr (:) []
>>> toList = Stream.fold Fold.toList

Convert a stream into a list in the underlying monad. The list can be consumed lazily in a lazy monad (e.g. Identity). In a strict monad (e.g. IO) the whole list is generated and buffered before it can be consumed.

Warning! working on large lists accumulated as buffers in memory could be very inefficient, consider using Streamly.Data.Array instead.

Note that this could a bit more efficient compared to Stream.fold Fold.toList, and it can fuse with pure list consumers.

>>> mapM f = Stream.sequence . fmap f

Apply a monadic function to each element of the stream and replace it with the output of the resulting action.

>>> s = Stream.fromList ["a", "b", "c"]
>>> Stream.fold Fold.drain $ Stream.mapM putStr s
abc

Take first n elements from the stream and discard the rest.

End the stream as soon as the predicate fails on an element.

Same as takeWhile but with a monadic predicate.

Like zipWith but using a monadic zipping function.

Stream a is evaluated first, followed by stream b, the resulting elements a and b are then zipped using the supplied zip function and the result c is yielded to the consumer.

If stream a or stream b ends, the zipped stream ends. If stream b ends first, the element a from previous evaluation of stream a is discarded.

>>> s1 = Stream.fromList [1,2,3]
>>> s2 = Stream.fromList [4,5,6]
>>> Stream.fold Fold.toList $ Stream.zipWith (+) s1 s2
[5,7,9]

Apply a stream of functions to a stream of values and flatten the results.

Note that the second stream is evaluated multiple times.

>>> crossApply = Stream.crossWith id

Definition:

>>> crossWith f m1 m2 = fmap f m1 `Stream.crossApply` m2

Note that the second stream is evaluated multiple times.

Given a Stream m a and Stream m b generate a stream with all possible combinations of the tuple (a, b).

Definition:

>>> cross = Stream.crossWith (,)

The second stream is evaluated multiple times. If that is not desired it can be cached in an Array and then generated from the array before calling this function. Caching may also improve performance if the stream is expensive to evaluate.

See cross for a much faster fused alternative.

Time: O(m x n)

Pre-release

unfoldMany unfold stream uses unfold to map the input stream elements to streams and then flattens the generated streams into a single output stream.

Like concatMap but uses an Unfold for stream generation. Unlike concatMap this can fuse the Unfold code with the inner loop and therefore provide many times better performance.

Given a stream value in the underlying monad, lift and join the underlying monad with the stream monad.

>>> concatEffect = Stream.concat . Stream.fromEffect
>>> concatEffect eff = Stream.concatMapM (\() -> eff) (Stream.fromPure ())

See also: concat, sequence

Map a stream producing function on each element of the stream and then flatten the results into a single stream.

>>> concatMap f = Stream.concatMapM (return . f)
>>> concatMap f = Stream.concat . fmap f
>>> concatMap f = Stream.unfoldMany (Unfold.lmap f Unfold.fromStream)

See unfoldMany for a fusible alternative.

Map a stream producing monadic function on each element of the stream and then flatten the results into a single stream. Since the stream generation function is monadic, unlike concatMap, it can produce an effect at the beginning of each iteration of the inner loop.

See unfoldMany for a fusible alternative.

Flatten a stream of streams to a single stream.

>>> concat = Stream.concatMap id

Pre-release

Same as concatIterateDfs but more efficient due to stream fusion.

Example, list a directory tree using DFS:

>>> f = Unfold.either Dir.eitherReaderPaths Unfold.nil
>>> input = Stream.fromPure (Left ".")
>>> ls = Stream.unfoldIterateDfs f input

Pre-release

Like unfoldIterateDfs but uses breadth first style traversal.

Pre-release

Like unfoldIterateBfs but processes the children in reverse order, therefore, may be slightly faster.

Pre-release

Generate a stream from an initial state, scan and concat the stream, generate a stream again from the final state of the previous scan and repeat the process.

Traverse the stream in depth first style (DFS). Map each element in the input stream to a stream and flatten, recursively map the resulting elements as well to a stream and flatten until no more streams are generated.

Example, list a directory tree using DFS:

>>> f = either (Just . Dir.readEitherPaths) (const Nothing)
>>> input = Stream.fromPure (Left ".")
>>> ls = Stream.concatIterateDfs f input

This is equivalent to using concatIterateWith StreamK.append.

Pre-release

Similar to concatIterateDfs except that it traverses the stream in breadth first style (BFS). First, all the elements in the input stream are emitted, and then their traversals are emitted.

Example, list a directory tree using BFS:

>>> f = either (Just . Dir.readEitherPaths) (const Nothing)
>>> input = Stream.fromPure (Left ".")
>>> ls = Stream.concatIterateBfs f input

Pre-release

Same as concatIterateBfs except that the traversal of the last element on a level is emitted first and then going backwards up to the first element (reversed ordering). This may be slightly faster than concatIterateBfs.

Apply a Fold repeatedly on a stream and emit the results in the output stream.

Definition:

>>> foldMany f = Stream.parseMany (Parser.fromFold f)

Example, empty stream:

>>> f = Fold.take 2 Fold.sum
>>> fmany = Stream.fold Fold.toList . Stream.foldMany f
>>> fmany $ Stream.fromList []
[]

Example, last fold empty:

>>> fmany $ Stream.fromList [1..4]
[3,7]

Example, last fold non-empty:

>>> fmany $ Stream.fromList [1..5]
[3,7,5]

Note that using a closed fold e.g. Fold.take 0, would result in an infinite stream on a non-empty input stream.

Like foldMany but evaluates the fold even if the fold did not receive any input, therefore, always results in a non-empty output even on an empty stream (default result of the fold).

Example, empty stream:

>>> f = Fold.take 2 Fold.sum
>>> fmany = Stream.fold Fold.toList . Stream.foldManyPost f
>>> fmany $ Stream.fromList []
[0]

Example, last fold empty:

>>> fmany $ Stream.fromList [1..4]
[3,7,0]

Example, last fold non-empty:

>>> fmany $ Stream.fromList [1..5]
[3,7,5]

Note that using a closed fold e.g. Fold.take 0, would result in an infinite stream without consuming the input.

Pre-release

Like foldMany but for the Refold type. The supplied action is used as the initial value for each refold.

Internal

Binary BFS style reduce, folds a level entirely using the supplied fold function, collecting the outputs as next level of the tree, then repeats the same process on the next level. The last elements of a previously folded level are folded first.

N-Ary BFS style iterative fold, if the input stream finished before the fold then it returns Left otherwise Right. If the fold returns Left we terminate.

Unimplemented

Compare two streams for equality

Compare two streams lexicographically.