| Safe Haskell | Safe |
|---|---|
| Language | Haskell2010 |
Control.Monad.Tokenizer.Class
Contents
Description
A monad class for writing pure tokenizers in an imperative-looking way.
Main idea: You walk through the input string like a turtle, and everytime
you find a token boundary, you call emit. If some specific kinds of tokens
should be suppressed, you can discard them instead (or filter afterwards).
An concrete instance of this class is provided in Control.Monad.Tokenizer.
Synopsis
- class Monad m => MonadTokenizer m where
- untilEOT :: MonadTokenizer m => m () -> m ()
- pop :: MonadTokenizer m => m Char
- walkWhile :: MonadTokenizer m => (Char -> Bool) -> m ()
- walkFold :: MonadTokenizer m => a -> (Char -> a -> Maybe a) -> m ()
- class Tokenizable t where
Monad class
class Monad m => MonadTokenizer m where Source #
A monad for turtle tokenization.
Methods
Proceed to the next character
Walk back to the previous character, unless it was discarded/emitted.
Peek the current character
Restore the state after the last emit/discard.
Break at the current position and emit the scanned token
Break at the current position and discard the scanned token
Have I reached the end of the input text?
lookAhead :: [Char] -> m Bool Source #
Check if the next input chars agree with the given string
Peek the current character and proceed
Instances
| Tokenizable t => MonadTokenizer (Tokenizer t) Source # | |
Defined in Control.Monad.Tokenizer | |
untilEOT :: MonadTokenizer m => m () -> m () Source #
Repeat a given tokenizer as long as the end of text is not reached
Utilities
pop :: MonadTokenizer m => m Char Source #
Peek the current character and proceed
walkWhile :: MonadTokenizer m => (Char -> Bool) -> m () Source #
Proceed as long as a given function succeeds
walkFold :: MonadTokenizer m => a -> (Char -> a -> Maybe a) -> m () Source #
Proceed as long as a given fold returns Just (generalization of walkWhile)
Text types
class Tokenizable t where Source #
Text types that can be split by the Tokenizer monad. In this module, instances are provided for String, strict Text, and lazy Text. If you are dealing with ASCII ByteStrings, you can find instances in the modules Control.Monad.Tokenizer.Char8.Strict and Control.Monad.Tokenizer.Char8.Lazy
Instances
| Tokenizable ByteString Source # | Assuming ASCII encoding |
Defined in Control.Monad.Tokenizer.Char8.Lazy Methods tnull :: ByteString -> Bool Source # thead :: ByteString -> Char Source # ttail :: ByteString -> ByteString Source # ttake :: Int -> ByteString -> ByteString Source # tdrop :: Int -> ByteString -> ByteString Source # tlower :: ByteString -> ByteString Source # | |
| Tokenizable ByteString Source # | Assuming ASCII encoding |
Defined in Control.Monad.Tokenizer.Char8.Strict Methods tnull :: ByteString -> Bool Source # thead :: ByteString -> Char Source # ttail :: ByteString -> ByteString Source # ttake :: Int -> ByteString -> ByteString Source # tdrop :: Int -> ByteString -> ByteString Source # tlower :: ByteString -> ByteString Source # | |
| Tokenizable Text Source # | |
| Tokenizable Text Source # | |
| Tokenizable [Char] Source # | |