Convert a lazy Text into a Producer of strict Texts

Stream text from stdin

Convert a Handle into a text stream using a text size determined by the good sense of the text library; note that this is distinctly slower than decideUtf8 (Pipes.ByteString.fromHandle h) but uses the system encoding and has other IO features

Stream text from a file in the simple fashion of Data.Text.IO

>>> runSafeT $ runEffect $ Text.readFile "hello.hs" >-> Text.map toUpper >-> hoist lift Text.stdout
MAIN = PUTSTRLN "HELLO WORLD"

Stream text to stdout

Unlike toHandle, stdout gracefully terminates on a broken output pipe.

Note: For best performance, it might be best just to use (for source (liftIO . putStr)) instead of (source >-> stdout) .

Convert a text stream into a Handle

Note: again, for best performance, where possible use (for source (liftIO . hPutStr handle)) instead of (source >-> toHandle handle).

Stream text into a file. Uses pipes-safe.

Apply a transformation to each Char in the stream

Map a function over the characters of a text stream and concatenate the results

(take n) only allows n individual characters to pass; contrast Pipes.Prelude.take which would let n chunks pass.

(drop n) drops the first n characters

Take characters until they fail the predicate

Drop characters until they fail the predicate

Only allows Chars to pass if they satisfy the predicate

Strict left scan over the characters

Transform a Pipe of Text into a Pipe of ByteStrings using UTF-8 encoding; encodeUtf8 = Pipes.Prelude.map TE.encodeUtf8 so more complex encoding pipes can easily be constructed with the functions in Data.Text.Encoding

Transform a Pipe of Strings into one of Text chunks

Transform a Pipes of Text chunks into one of Strings

toCaseFold, toLower, toUpper and stripStart are standard Text utilities, here acting as Text pipes, rather as they would on a lazy text

lowercase incoming Text

uppercase incoming Text

Remove leading white space from an incoming succession of Texts

Fold a pure Producer of strict Texts into a lazy Text

Fold an effectful Producer of strict Texts into a lazy Text

Note: toLazyM is not an idiomatic use of pipes, but I provide it for simple testing purposes. Idiomatic pipes style consumes the chunks immediately as they are generated instead of loading them all into memory.

Reduce the text stream using a strict left fold over characters

Retrieve the first Char

Retrieve the last Char

Determine if the stream is empty

Count the number of characters in the stream

Fold that returns whether Any received Chars satisfy the predicate

Fold that returns whether All received Chars satisfy the predicate

Return the maximum Char within a text stream

Return the minimum Char within a text stream (surely very useful!)

Find the first element in the stream that matches the predicate

Index into a text stream

Store a tally of how many segments match the given Text

Consume the first character from a stream of Text

next either fails with a Left if the Producer has no more characters or succeeds with a Right providing the next character and the remainder of the Producer.

Draw one Char from a stream of Text, returning Left if the Producer is empty

Push back a Char onto the underlying Producer

peekChar checks the first Char in the stream, but uses unDrawChar to push the Char back

peekChar = do
    x <- drawChar
    case x of
        Left  _  -> return ()
        Right c -> unDrawChar c
    return x

Check if the underlying Producer has no more characters

Note that this will skip over empty Text chunks, unlike isEndOfInput from pipes-parse, which would consider an empty Text a valid bit of input.

isEndOfChars = liftM isLeft peekChar

Splits a Producer after the given number of characters

Split a text stream in two, where the first text stream is the longest consecutive group of text that satisfy the predicate

Split a text stream in two, where the first text stream is the longest consecutive group of characters that don't satisfy the predicate

Improper lens that splits after the first group of equivalent Chars, as defined by the given equivalence relation

Improper lens that splits after the first succession of identical Char s

Improper lens that splits a Producer after the first word

Unlike words, this does not drop leading whitespace

An improper lens into a stream of ByteString expected to be UTF-8 encoded; the associated stream of Text ends by returning a stream of ByteStrings beginning at the point of failure.

Use a Codec as a pipes-style Lens into a byte stream; the available Codec s are utf8, utf16_le, utf16_be, utf32_le, utf32_be . The Codec concept and the individual Codec definitions follow the enumerator and conduit libraries.

Utf8 is handled differently in this library -- without the use of unsafePerformIO &co to catch Text exceptions; but the same 'mypipe ^. codec utf8' interface can be used. 'mypipe ^. decodeUtf8' should be the same, but has a somewhat more direct and thus perhaps better implementation.

Reduce a byte stream to a corresponding stream of ascii chars, returning the unused ByteString upon hitting the rare un-latinizable byte.

Reduce a byte stream to a corresponding stream of ascii chars, returning the unused ByteString upon hitting an un-ascii byte.

Reduce as much of your stream of Text actually is iso8859 or latin1 to a byte stream, returning the rest of the Text upon hitting any non-latin Char

ascii and latin encodings only represent a small fragment of Text; thus we cannot use the pipes Lens style to work with them. Rather we simply define functions each way.

encodeAscii : Reduce as much of your stream of Text actually is ascii to a byte stream, returning the rest of the Text at the first non-ascii Char

Split a text stream into FreeT-delimited text streams of fixed size

Split a text stream into sub-streams delimited by characters that satisfy the predicate

Split a text stream using the given Char as the delimiter

Split a text stream into FreeT-delimited lines

Split a text stream into FreeT-delimited words

Intersperse a Char in between the characters of stream of Text

Improper isomorphism between a Producer of ByteStrings and Word8s

intercalate concatenates the FreeT-delimited text streams after interspersing a text stream in between them

Join FreeT-delimited lines into a text stream

Join FreeT-delimited words into a text stream

A specific character encoding.