Return the next token. This is the function used by Happy in the parser.

lexer k = lexToken >>= k

This is the initial state for parsing a regular, non-literate file.

The layout state. Entered when we see a layout keyword (withLayout) and exited either when seeing an open brace (openBrace) or at the next token (newLayoutContext).

Update: we don't use braces for layout anymore.

We enter this state from newLayoutContext when the token following a layout keyword is to the left of (or at the same column as) the current layout context. Example:

data Empty : Set where
foo : Empty -> Nat

Here the second line is not part of the where clause since it is has the same indentation as the data definition. What we have to do is insert an empty layout block {} after the where. The only thing that can happen in this state is that emptyLayout is executed, generating the closing brace. The open brace is generated when entering by newLayoutContext.

This state is entered at the beginning of each line. You can't lex anything in this state, and to exit you have to check the layout rule. Done with offsideRule.

This state can only be entered by the parser. In this state you can only lex the keywords using, hiding, renaming and to. Moreover they are only keywords in this particular state. The lexer will never enter this state by itself, that has to be done in the parser.

This is the main lexing function generated by Alex.