Copyright	(c) 2008 Utrecht University
License	All rights reserved
Maintainer	stefan@cs.uu.nl
Stability	provisional
Portability	portable
Safe Haskell	Safe
Language	Haskell98

CCO.Lexing

Contents

Lexers
Derived lexers
Ignoring recognised input
Symbols
Lexing
Utilities

Description

A library of lexer combinators that expose their functionality through an Applicative interface.

Synopsis

Lexers

data Lexer a Source

Type of lexers that produce tokens of type a.

Instances

Functor Lexer Source
Applicative Lexer Source
Alternative Lexer Source

satisfy :: (Char -> Bool) -> Lexer Char Source

A Lexer that recognises any character that satisfies a specified predicate.

message :: String -> Lexer a Source

A Lexer that will produce a specified error message.

Derived lexers

anyChar :: Lexer Char Source

The trivial Lexer that recognises any character.

anyCharFrom :: [Char] -> Lexer Char Source

A Lexer that recognises any character that appears in a given list.

anyCharBut :: [Char] -> Lexer Char Source

A Lexer that recognises any character that does not appear in a given list.

range :: (Char, Char) -> Lexer Char Source

A Lexer that recognises any character that appears in a given range. More efficent than \(low, up) -> anyCharFrom [low .. up].

notInRange :: (Char, Char) -> Lexer Char Source

A Lexer that recognises any character that does not appear in a given range.

char :: Char -> Lexer Char Source

A Lexer that recognises a specified character.

string :: String -> Lexer String Source

A Lexer that recognises a specified String.

space :: Lexer Char Source

A Lexer that recognises a whitespace character

lower :: Lexer Char Source

A Lexer that recognises lower-case alphabetic characters.

upper :: Lexer Char Source

A Lexer that recognises upper-case or title-case alphabetic characters.

letter :: Lexer Char Source

A Lexer that recognises alphabetic characters. Equivalent to alpha.

alpha :: Lexer Char Source

A Lexer that recognises alphabetic characters. Equivalet to letter.

alphaNum :: Lexer Char Source

A Lexer that recognises alphabetic or numeric characters.

digit :: Lexer Char Source

A Lexer that recognises a digit.

digit_ :: Lexer Int Source

A Lexer that recognises a digit and tokenises it as an Int.

binDigit :: Lexer Char Source

A Lexer that recognises a binary digit.

binDigit_ :: Lexer Int Source

A Lexer that recognises a binary digit and tokenises it as an Int.

octDigit :: Lexer Char Source

A Lexer that recognises an octal digit.

octDigit_ :: Lexer Int Source

A Lexer that recognises an octal digit and tokenises it as an Int.

hexDigit :: Lexer Char Source

A 'Lexer that recognises a hexedecimal digit.

hexDigit_ :: Lexer Int Source

A Lexer that recognises a hexadecimal digit and tokenises it as an Int.

Ignoring recognised input

ignore :: Lexer a -> Lexer b Source

Produces a Lexer that recognises the same inputs as a given underlying Lexer, but that does not result in any tokenisation.

The input recognised by a Lexer constructed with ignore is simply ignored when the Lexer is used to turn a stream of characters into a stream of LexicalUnits.

Mainly used to suppress the generation of tokens for lexemes that constitute-- lexical units like comments and whitespace.

Symbols

data LexicalUnit a Source

The type of lexical units.

Constructors

Token a Pos String String	A tokenised lexeme: its token, its position, the characters it consists of, and its its trailing characters in the input stream.
Error Pos String String	An invalid lexeme: its position, the characters it consists of, and its trailing characters in the input stream.
Msg String Pos String String	An invalid lexeme, labeled by a custom error message: the message, its position, the characters it consists of, and its trailing characters in the input stream.

data Symbols a Source

The type of streams of symbols described by tokens of type a.

Constructors

Symbols Source [LexicalUnit a]

Lexing

lex :: Lexer a -> Source -> String -> Symbols a Source

Runs a lexer on a specified input stream.

Utilities

tokens :: Symbols a -> [(SourcePos, a)] Source

Retrieves all tokens together with their source positions from a list of Symbols.

tokens_ :: Symbols a -> [a] Source

Retrieves all tokens from a list of Symbols.