| Safe Haskell | Safe-Inferred |
|---|---|
| Language | Haskell2010 |
Parser.Tokeniser
Description
- Tokeniser
- A data structure representing a sequence of tokens, for internal use in the parser
Synopsis
- data Tokeniser' char token err t
- data Tokens' token
- add_token :: Line_and_char -> token -> Tokeniser' char token err ()
- current_line_and_char :: Tokens' token -> Line_and_char
- delete_char :: Tokeniser' char token err ()
- gather_token :: (char -> Maybe Char) -> (String -> token) -> Tokeniser' char token err ()
- get_char :: Int -> Tokeniser' char token err (Maybe char)
- get_line_and_char :: Tokeniser' char token err Line_and_char
- get_token :: Tokens' token -> Maybe token
- take_token :: (token -> Maybe t) -> Tokens' token -> Maybe (t, Tokens' token)
- tokenisation_error :: (Line_and_char -> err) -> Tokeniser' char token err t
- tokenise :: (Char -> char) -> (char -> Line_and_char -> Line_and_char) -> Tokeniser' char token err () -> String -> Either Error (Either err (Tokens' token))
- tokens_ended :: Tokens' token -> Bool
Documentation
data Tokeniser' char token err t Source #
A tokeniser that works with any kind of custom characters, tokens and errors. The custom character type is necessary if you want to classify characters according to their behavior before tokenisation - for example, wrap all operators, letters, delimiters or digits in the same constructor to simplify pattern matching.
Instances
| Monad (Tokeniser' char token err) Source # | |
Defined in Parser.Tokeniser Methods (>>=) :: Tokeniser' char token err a -> (a -> Tokeniser' char token err b) -> Tokeniser' char token err b # (>>) :: Tokeniser' char token err a -> Tokeniser' char token err b -> Tokeniser' char token err b # return :: a -> Tokeniser' char token err a # | |
| Functor (Tokeniser' char token err) Source # | |
Defined in Parser.Tokeniser Methods fmap :: (a -> b) -> Tokeniser' char token err a -> Tokeniser' char token err b # (<$) :: a -> Tokeniser' char token err b -> Tokeniser' char token err a # | |
| Applicative (Tokeniser' char token err) Source # | |
Defined in Parser.Tokeniser Methods pure :: a -> Tokeniser' char token err a # (<*>) :: Tokeniser' char token err (a -> b) -> Tokeniser' char token err a -> Tokeniser' char token err b # liftA2 :: (a -> b -> c) -> Tokeniser' char token err a -> Tokeniser' char token err b -> Tokeniser' char token err c # (*>) :: Tokeniser' char token err a -> Tokeniser' char token err b -> Tokeniser' char token err b # (<*) :: Tokeniser' char token err a -> Tokeniser' char token err b -> Tokeniser' char token err a # | |
A sequence of tokens with locations. For internal use in the parser.
add_token :: Line_and_char -> token -> Tokeniser' char token err () Source #
Add the token to the output. Note that the order of adding tokens is important.
current_line_and_char :: Tokens' token -> Line_and_char Source #
Get the location of the first token or, if there are none, the end of file. For internal use in the parser.
delete_char :: Tokeniser' char token err () Source #
Delete the first character from the remaining text. Automatically updates the location.
gather_token :: (char -> Maybe Char) -> (String -> token) -> Tokeniser' char token err () Source #
Add a token that consists of several characters - for example, an operator, a word or a natural number. You have to provide a function that recognises suitable characters and a function that transforms the resulting string into a token.
get_char :: Int -> Tokeniser' char token err (Maybe char) Source #
Take a look at a character without deleting it. Returns Nothing if the index is negative or if the remaining text is
too short.
get_line_and_char :: Tokeniser' char token err Line_and_char Source #
Get the current location of the tokeniser.
get_token :: Tokens' token -> Maybe token Source #
Get the first token without deleting it. For internal use in the parser.
take_token :: (token -> Maybe t) -> Tokens' token -> Maybe (t, Tokens' token) Source #
Recognises tokens that fit a certain pattern and transforms them into something more useful - for example, a string or an
integer. Returns Nothing if the first token does not fit the pattern, and returns the transformed token and the rest of
the sequence if it does fit. For internal use in the parser.
tokenisation_error :: (Line_and_char -> err) -> Tokeniser' char token err t Source #
Throw a tokenisation error at the current location.
tokenise :: (Char -> char) -> (char -> Line_and_char -> Line_and_char) -> Tokeniser' char token err () -> String -> Either Error (Either err (Tokens' token)) Source #
Tokenise the text. For internal use in the parser.
tokens_ended :: Tokens' token -> Bool Source #
Check whether the sequence of tokens has ended. For internal use in the parser.