The type of an iterator-enumerator, which transcodes data from some input type tIn to some output type tOut. An Inum acts as an Iter when consuming data, then acts as an enumerator when feeding transcoded data to another Iter.

At a high level, one can think of an Inum as a function from Iters to IterRs, where an Inum's input and output types are different. A simpler-seeming alternative to Inum might have been:

 type Inum' tIn tOut m a = Iter tOut m a -> Iter tIn m a

In fact, given an Inum object inum, it is possible to construct a function of type Inum' with (inum .|). But sometimes one might like to concatenate Inums. For instance, consider a network protocol that changes encryption or compression modes midstream. Transcoding is done by Inums. To change transcoding methods after applying an Inum to an iteratee requires the ability to "pop" the iteratee back out of the Inum so as to be able to hand it to another Inum. Inum's return type (Iter tIn m (IterR tOut m a) as opposed to Iter tIn m a) allows the monadic bind operator >>= to accomplish this popping in conjunction with the tryRI and reRunIter functions.

All Inums must obey the following two rules.

1. An Inum may never feed a chunk with the EOF flag set to it's target Iter. Instead, upon receiving EOF, the Inum should simply return the state of the inner Iter (this is how "popping" the iteratee back out works--If the Inum passed the EOF through to the Iter, the Iter would stop requesting more input and could not be handed off to a new Inum).
2. An Inum must always return the state of its target Iter. This is true even when the Inum fails, and is why the Fail state contains a Maybe a field.

In addition to returning when it receives an EOF or fails, an Inum should return when the target Iter returns a result or fails. An Inum may also unilaterally return the state of the iteratee at any earlier point, for instance if it has reached some logical message boundary (e.g., many protocols finish processing headers upon reading a blank line).

Inums are generally constructed with one of the mkInum or mkInumM functions, which hide most of the error handling details and ensure the above rules are obeyed. Most Inums are polymorphic in the last type, a, in order to work with iteratees returning any type. There isn't much reason for an Inum to care about the type a. Had this module used the Rank2Types Haskell extension, it would define Inum as:

 type Inum tIn tOut m = forall a. Iter tOut m a
                               -> Iter tIn m (IterR tOut m a)

An Onum t m a is just an Inum in which the input is ()--i.e., Inum () t m a--so that there is no meaningful input data to transcode. Such an enumerator is called an outer enumerator, because it must produce the data it feeds to Iters by either executing actions in monad m, or from its own internal pure state (as for enumPure).

As with Inums, an Onum should under no circumstances ever feed a chunk with the EOF bit set to its Iter argument. When the Onum runs out of data, it must simply return the current state of the Iter. This way more data from another source can still be fed to the iteratee, as happens when enumerators are concatenated with the cat function.

Onums should generally be constructed using the mkInum or mkInumM function, just like Inums, the only difference being that for an Onum the input type is (), so executing Iters to consume input will be of little use.

Run an Onum on an Iter. This is the main way of actually executing IO with Iters. |$ is a type-restricted version of the following code, in which inum must be an Onum:

  inum |$ iter = run (inum .| iter)
  infixr 2 |$

.|$ is a variant of |$ that allows you to apply an Onum from within an Iter monad. This is often useful in conjuction with enumPure, if you want to parse at some coarse-granularity (such as lines), and then re-parse the contents of some coarser-grained parse unit. For example:

     rawcommand <- lineI
     command <- enumPure rawcommand .|$ parseCommandI
     return Request { cmd = command, rawcmd = rawcommand }

.|$ has the same fixity as |$, namely:

 infixr 2 .|$

Note the important distinction between (.|$) and (.|). (.|$) runs an Onum and does not touch the current input, while (.|) pipes the current input through an Inum. For instance, to send the contents of a file to standard output (regardless of the current input), you must say enumFile ".signature" .|$ stdoutI. But to take the current input, compress it, and send the result to standard output, you must use .|, as in inumGzip .| stdoutI.

As suggested by the types, enum .|$ iter is sort of equivalent to lift (enum |$ iter), except that the latter will call throw on failures, causing language-level exceptions that cannot be caught within the outer Iter. Thus, it is better to use .|$ than lift (... |$ ...), though in the less general case of the IO monad, enum .|$ iter is equivalent to liftIO (enum |$ iter) as illustrated by the following examples:

 -- Catches exception, because .|$ propagates failure through the outer
 -- Iter Monad, where it can still be caught.
 apply1 :: IO String
 apply1 = enumPure "test1" |$ iter `catchI` handler
     where
       iter = enumPure "test2" .|$ fail "error"
       handler (SomeException _) _ = return "caught error"
 
 -- Does not catch error.  |$ turns the Iter failure into a language-
 -- level exception, which can only be caught in the IO Monad.
 apply2 :: IO String
 apply2 = enumPure "test1" |$ iter `catchI` handler
     where
       iter = lift (enumPure "test2" |$ fail "error")
       handler (SomeException _) _ = return "caught error"
 
 -- Catches the exception, because liftIO uses the IO catch function to
 -- turn language-level exceptions into monadic Iter failures.  (By
 -- contrast, lift works in any Monad, so cannot do this in apply2.)
 -- This example illustrates how liftIO is not equivalent to lift.
 apply3 :: IO String
 apply3 = enumPure "test1" |$ iter `catchI` handler
     where
       iter = liftIO (enumPure "test2" |$ fail "error")
       handler (SomeException _) _ = return "caught error"

Concatenate the outputs of two enumerators. For example, enumFile "file1" `cat` enumFile "file2" produces an Onum that outputs the concatenation of files "file1" and "file2". Unless the first Inum fails, cat always invokes the second Inum, as the second Inum may have monadic side-effects that must be executed even when the Iter has already finished. See lcat if you want to stop when the Iter no longer requires input. If you want to continue executing even in the event of an InumFail condition, you can wrap the first Inum with inumCatch and invoke resumeI from within the exception handler.

cat (and lcat, described below) are useful in right folds. Say, for instance, that files is a list of files you wish to concatenate. You can use a construct such as:

  catFiles :: (MonadIO m) => [FilePath] -> Onum L.ByteString m a
  catFiles files = foldr (cat . enumFile) inumNull files

Note the use of inumNull as the starting value for foldr. This is not to be confused with inumNop. inumNull acts as a no-op for concatentation, producing no output analogously to /dev/null. By contrast inumNop is the no-op for fusing (see |. and .| below) because it passes all data through untouched.

cat has fixity:

 infixr 3 `cat`

Lazy cat. Like cat, except that it does not run the second Inum if the Iter is no longer active after completion of the first Inum. Also has fixity infixr 3 `lcat`.

Left-associative pipe operator. Fuses two Inums when the output type of the first Inum is the same as the input type of the second. More specifically, if inum1 transcodes type tIn to tOut and inum2 transcodes tOut to tOut2, then inum1 |. inum2 produces a new Inum that transcodes from tIn to tOut2.

Typically types i and iR are Iter tOut2 m a and IterR tOut2 m a, respectively, in which case the second argument and result of |. are also Inums.

This function is equivalent to:

  outer |. inner = \iter -> outer .| inner iter
  infixl 4 |.

But if you like point-free notation, think of it as outer |. inner = (outer .|) . inner, or better yet (|.) = (.) . (.|).

Right-associative pipe operator. Fuses an Inum that transcodes tIn to tOut with an Iter taking input type tOut to produce an Iter taking input type tIn. If the Iter is still active when the Inum terminates (either normally or through an exception), then .| sends it an EOF.

Has fixity:

 infixr 4 .|

Catches errors thrown by an Inum, or a set of fused Inums. Note that only errors in Inums that are lexically within the scope of the argument to inumCatch will be caught. For example:

 inumBad :: (ChunkData t, Monad m) => Inum t t m a
 inumBad = mkInum $ fail "inumBad"
 
 skipError :: (ChunkData tIn, MonadIO m) =>
              SomeException
           -> IterR tIn m (IterR tOut m a)
           -> Iter tIn m (IterR tOut m a)
 skipError e iter = do
   liftIO $ hPutStrLn stderr $ "skipping error: " ++ show e
   resumeI iter

 -- Throws an exception, because inumBad was fused outside the argument
 -- to inumCatch.
 test1 :: IO ()
 test1 = inumCatch (enumPure "test") skipError |. inumBad |$ nullI
 
 -- Does not throw an exception, because inumBad fused within the
 -- argument to inumCatch.
 test2 :: IO ()
 test2 = inumCatch (enumPure "test" |. inumBad) skipError |$ nullI
 
 -- Again no exception, because inumCatch is wrapped around inumBad.
 test3 :: IO ()
 test3 = enumPure "test" |. inumCatch inumBad skipError |$ nullI

Note that `inumCatch` has the default infix precedence (infixl 9 `inumcatch`), which binds more tightly than any concatenation or fusing operators.

As noted for catchI, exception handlers receive both the exception thrown and the failed IterR. Particularly in the case of inumCatch, it is important to re-throw exceptions by re-executing the failed Iter with reRunIter, not passing the exception itself to throwI. That way, if the exception is re-caught, resumeI will continue to work properly. For example, to copy two files to standard output and ignore file not found errors but re-throw any other kind of error, you could use the following:

  resumeTest :: IO ()
  resumeTest = doFile "file1" `cat` doFile "file2" |$ stdoutI
      where
        doFile path = inumCatch (enumFile' path) $ \err r ->
                        if isDoesNotExistError err
                          then verboseResumeI r
                          else reRunIter r

Execute some cleanup action when an Inum finishes.

Execute some cleanup action if an Inum fails. Does not execute the action if the Iter (or some inner Inum) fails. Has the same scoping rules as inumCatch.

Used in an exception handler, after an Inum failure, to resume processing of the Iter by the next enumerator in a cated series. See inumCatch for an example.

Like resumeI, but if the Iter is resumable, also prints an error message to standard error before resuming.

A ResidHandler specifies how to handle residual data in an Inum. Typically, when an Inum finishes executing, there are two kinds of residual data. First, the Inum itself (in its role as an iteratee) may have left some unconsumed data. Second, the target Iter being fed by the Inum may have some resitual data, and this data may be of a different type. A ResidHandler allows this residual data to be adjusted by untranslating the residual data of the target Iter and sticking the result back into the Inum's residual data.

The two most common ResidHandlers are pullupResid (to pull the target Iter's residual data back up to the Inum as is), and id (to do no adjustment of residual data).

ResidHandlers are used by the mkInumC function, and by the passCtl CtlHandler.

A control handler maps control requests to IterR results. Generally the type parameter m1 is Iter t' m.

Create a stateless Inum from a "codec" Iter that transcodes the input type to the output type. The codec is invoked repeately until one of the following occurs:

1. The input is at an EOF marker AND the codec returns null data. (Onums are always fed EOF, but other Inums might have reason to return mempty data.)
2. The codec throws an exception. If the exception is an EOF exception--thrown either by throwEOFI, or by some IO action inside liftIO--this is considered normal termination, and is the normal way for a codec to cause the Inum to return. If the exception is of any other type, then the Inum will further propagate the exception as an Inum failure.
3. The underlying target Iter either returns a result or throws an exception.

mkInumC requires two other arguments before the codec. First, a ResidHandler allows residual data to be adjusted between the input and output Iter monads. Second, a CtlHandler specifies a handler for control requests. For example, to pass up control requests and ensure no residual data is lost when the Inum is fused to an Iter, the inumConcat function given previously for mkInum at #mkInumExample could be re-written:

 inumConcat :: (Monad m) => Inum [L.ByteString] L.ByteString m a
 inumConcat = mkInumC reList (passCtl reList) iterConcat
     where reList (a, b) = (b:a, mempty)

Create an Inum based on an Iter that transcodes the input to the output type. This is a simplified version of mkInumC that rejects all control requests and does not adjust residual data.

 mkInum = mkInumC id noCtl

A simplified version of mkInum that passes all control requests to enclosing enumerators. It requires a ResidHandler to describe how to adjust residual data. (E.g., use pullupResid when tIn and tOut are the same type.)

 mkInumP adj = mkInumC adj (passCtl adj)

Bracket an Inum with a start and end function, which can be used to acquire and release a resource, must like the IO monad's bracket function. For example:

 enumFile :: (MonadIO m, ChunkData t, LL.ListLikeIO t e) =>
             FilePath -> Onum t m a
 enumFile path = inumBracket (liftIO $ openBinaryFile path ReadMode)
                             (liftIO . hClose)
                             enumHandle

pullupResid (a, b) = (mappend a b, mempty). See ResidHandler.

Reject all control requests.

Pass all control requests through to the enclosing Iter monad. The ResidHandler argument says how to adjust residual data, in case some enclosing CtlHandler decides to flush pending input data, it is advisable to un-translate any data in the output type tOut back to the input type tIn.

Create a CtlHandler given a function of a particular control argument type and a fallback CtlHandler to run if the argument type does not match. consCtl is used to chain handlers, with the rightmost handler being either noCtl or passCtl.

For example, to create a control handler that implements seek on SeekC requests, returns the size of the file on SizeC requests, and passes everything else out to the enclosing enumerator (if any), you could use the following:

 fileCtl :: (ChunkData t, MonadIO m) => Handle -> CtlHandler (Iter () m) t m a
 fileCtl h = (mkFlushCtl $ (SeekC mode pos) -> liftIO (hSeek h mode pos))
             `consCtl` (mkCtl $ SizeC -> liftIO (hFileSize h))
             `consCtl` passCtl id

Has fixity:

 infixr 9 `consCtl`

Make a control function suitable for use as the first argument to consCtl.

Like mkCtl, except that it flushes all input and clears the EOF flag in both Iter monads after executing the control function.

Like runIterMC, but only for IterM--may return IterC.

Run an Iter just like runIter, but then keep stepping the result for as long as it is in the IterM or IterC state (using the supplied CtlHandler for IterC states). Inums should generally use this function or runIterM in preference to runIter, as it is convenient if Inums avoid ever returning IterRs in the IterM state.

Takes an Inum that might return IterRs in the IterM state (which is considered impolite--see runIterMC) and transforms it into an Inum that never returns IterRs in the IterM state.

inumNop passes all data through to the underlying Iter. It acts as a no-op when fused to other Inums with |. or when fused to Iters with .|.

inumNop is particularly useful for conditionally fusing Inums together. Even though most Inums are polymorphic in the return type, this library does not use the Rank2Types extension, which means any given Inum must have a specific return type. Here is an example of incorrect code:

 let enum = if debug then base_enum |. inumStderr else base_enum -- Error

This doesn't work because base_enum cannot have the same type as (base_enum |. inumStderr). Instead, you can use the following:

 let enum = base_enum |. if debug then inumStderr else inumNop

inumNull feeds empty data to the underlying Iter. It pretty much acts as a no-op when concatenated to other Inums with cat or lcat.

There may be cases where inumNull is required to avoid deadlock. In an expression such as enum |$ iter, if enum immediately blocks waiting for some event, and iter immediately starts out triggering that event before reading any input, then to break the deadlock you can re-write the code as cat inumNull enum |$ iter.

Feed pure data to an Iter.

Type-restricted version of inumPure.

Repeat an Inum until the input receives an EOF condition, the Iter no longer requires input, or the Iter is in an unhandled IterC state (which presumably will continue to be unhandled by the same Inum, so no point in executing it again).

A monad in which to define the actions of an Inum tIn tOut m a. Note InumM tIn tOut m a is a Monad of kind * -> *, where a is the (almost always parametric) return type of the Inum. A fifth type argument is required for monadic computations of kind *, e.g.:

 seven :: InumM tIn tOut m a Int
 seven = return 7

Another important thing to note about the InumM monad, as described in the documentation for mkInumM, is that you must call lift twice to execute actions in monad m, and you must use the liftI function to execute actions in monad Iter t m a.

Build an Inum out of an InumM computation. If you run mkInumM inside the Iter tIn m monad (i.e., to create an enumerator of type Inum tIn tOut m a), then the InumM computation will be in a Monad of type Iter t tm where tm is a transformed version of m. This has the following two consequences:

• If you wish to execute actions in monad m from within your InumM computation, you will have to apply lift twice (as in lift $ lift action_in_m) rather than just once.
• If you need to execute actions in the Iter t m monad, you will have to lift them with the liftI function.

The InumM computation you construct can feed output of type tOut to the target Iter (which is implicitly contained in the monad state), using the ifeed, ipipe, and irun functions.

A variant of mkInumM that sets AutoEOF and AutoDone to True by default. (Equivalent to calling setAutoEOF True >> setAutoDone True as the first thing inside mkInumM.)

Set the control handler an Inum should use from within an InumM computation. (The default is noCtl.)

Set the AutoEOF flag within an InumM computation. If this flag is True, handle IterEOF exceptions like a normal but immediate termination of the Inum. If this flag is False (the default), then IterEOF exceptions must be manually caught or they will terminate the thread.

Set the AutoDone flag within an InumM computation. When True, the Inum will immediately terminate as soon as the Iter it is feeding enters a non-active state (i.e., Done or a failure state). If this flag is False (the default), the InumM computation will need to monitor the results of the ifeed, ipipe, and irun functions to ensure the Inum terminates when one of these functions returns False.

Add a cleanup action to be executed when the Inum finishes, or, if used in conjunction with the withCleanup function, when the innermost enclosing withCleanup action finishes.

Run an InumM with some cleanup action in effect. The cleanup action specified will be executed when the main action returns, whether normally, through an exception, because of the AutoDone or AutoEOF flags, or because idone is invoked.

Note withCleanup also defines the scope of actions added by the addCleanup function. In other words, given a call such as withCleanup cleaner1 main, if main invokes addCleanup cleaner2, then both cleaner1 and cleaner2 will be executed upon main's return, even if the overall Inum has not finished yet.

Used from within the InumM monad to feed data to the target Iter. Returns False if the target Iter is still active and True if the iter has finished and the Inum should also return. (If the autoDone flag is True, then ifeed, ipipe, and irun will never actually return True, but instead just immediately run cleanup functions and exit the Inum when the target Iter stops being active.)

A variant of ifeed that throws an exception of type IterEOF if the data being fed is null. Convenient when reading input with a function (such as Data.ListLike's hget) that returns 0 bytes instead of throwing an EOF exception to indicate end of file. For instance, the main loop of enumFile could be implemented as:

  irepeat $ liftIO (LL.hGet h defaultChunkSize) >>= ifeed1

Apply another Inum to the target Iter from within the InumM monad. As with ifeed, returns True when the Iter is finished.

Note that the applied Inum must handle all control requests. (In other words, ones it passes on are not caught by whatever handler is installed by setCtlHandler, but if the Inum returns the IterR in the IterC state, as inumPure does, then requests will be handled.)

Apply an Onum (or Inum of an arbitrary, unused input type) to the Iter from within the InumM monad. As with ifeed, returns True when the Iter is finished.

Repeats an action until the Iter is done or an EOF error is thrown. (Also stops if a different kind of exception is thrown, in which case the exception propagates further and may cause the Inum to fail.) irepeat sets both the AutoEOF and AutoDone flags to True.

If the target Iter being fed by the Inum is no longer active (i.e., if it is in the Done state or in an error state), this funciton pops the residual data out of the Iter and returns it. If the target is in any other state, returns mempty.

Immediately perform a successful exit from an InumM monad, terminating the Inum and returning the current state of the target Iter. Can be used to end an irepeat loop. (Use throwI ... for an unsuccessful exit.)