ace-0.6: Attempto Controlled English parser and printer

Safe HaskellNone
LanguageHaskell98

ACE.Tokenizer

Description

Tokenizer for ACE. Tokens retain source locations (line and column).

Synopsis

Documentation

tokenize :: Text -> Either String [Token] Source

Tokenize some complete ACE text.

tokenizer :: Parser ([Token], (Int, Int)) Source

The tokenizer.

token :: (Int, Int) -> Parser (Token, (Int, Int)) Source

Parse a token.

number :: (Int, Int) -> Parser (Token, (Int, Int)) Source

Parse a number.

quotedString :: (Int, Int) -> Parser (Token, (Int, Int)) Source

Parse a quoted string, "foobar".

period :: (Int, Int) -> Parser (Token, (Int, Int)) Source

Parse a period ".".

comma :: (Int, Int) -> Parser (Token, (Int, Int)) Source

Parse a comma ",".

questionMark :: (Int, Int) -> Parser (Token, (Int, Int)) Source

Parse a question mark "?".

word :: (Int, Int) -> Parser (Token, (Int, Int)) Source

Parse a word, which is any sequence of non-whitespace words containing none of the other token characters.

genitive :: (Int, Int) -> Parser (Maybe (Token, (Int, Int))) Source

Parse the Saxon genitive ' or 's. This is ran after parsing every token, but is expected to fail most of the time.

manyWithPos :: (Monad m, Alternative m) => ((t, t1) -> m (a, (t, t1))) -> ((t, t1) -> m (Maybe (a, (t, t1)))) -> (t, t1) -> m ([a], (t, t1)) Source

Like many, but retains the current source position and supports postfix-parsing of the genitive apostrophe.

spaces :: (Int, Int) -> Parser (Int, Int) Source

Skip spaces (space, newline, tab (=4 spaces)) and keep positioning information up to date.