Documentation

Pencil's Page AST.

Pencil's tokens for content.

Convert Tokens to PNode AST.

>>> transform [TokText "hello", TokText "world"]
[PText "hello",PText "world"]

>>> transform [TokIf "title", TokEnd]
[PIf "title" []]

>>> transform [TokIf "title", TokText "hello", TokText "world", TokEnd]
[PIf "title" [PText "hello",PText "world"]]

${if(title)}
  ${for(posts)}
    world
  ${end}
${end}

>>> transform [TokIf "title", TokFor "posts", TokText "world", TokEnd, TokEnd]
[PIf "title" [PFor "posts" [PText "world"]]]

begin
now
${if(title)}
  hello
  world
  ${if(body)}
    ${body}
    ${someothervar}
    wahh
  ${end}
  final
  thing
${end}
the
lastline

>>> transform [TokText "begin", TokText "now", TokIf "title", TokText "hello", TokText "world", TokIf "body", TokVar "body", TokVar "someothervar", TokText "wahh", TokEnd, TokText "final", TokText "thing", TokEnd, TokText "the", TokText "lastline"]
[PText "begin",PText "now",PIf "title" [PText "hello",PText "world",PIf "body" [PVar "body",PVar "someothervar",PText "wahh"],PText "final",PText "thing"],PText "the",PText "lastline"]

<!--PREAMBLE
foo: bar
do:
  - re
  - me
-->
Hello world ${foo}

>>> transform [TokPreamble "foo: bar\ndo:\n  - re\n  -me", TokText "Hello world ", TokVar "foo"]
[PPreamble "foo: bar\ndo:\n  - re\n  -me",PText "Hello world ",PVar "foo"]

Converts Tokens, which is just the raw list of parsed tokens, into PNodes which are the tree-structure expressions (i.e. if/for nesting)

This function works by using a stack to keep track of where we are for nested expressions such as if and for statements. When a token that starts a nesting is found (like a TokIf), a "meta" expression (PMetaIf) is pushed into the stack. When we finally see an end token (TokEnd), we pop all the expressions off the stack until the first meta tag (e.g PMetaIf) is reached. All the expressions popped off are now known to be nested inside that if statement.

Pop nodes until we hit a If/For statement. Return pair (constructor found, nodes popped, remaining stack)

Helper for popNodes.

Render nodes as string.

Render node as string.

Render tokens.

Render token.

Parse text.

Parse everything.

>>> parse parseEverything "" "Hello ${man} and ${woman}."
Right [TokText "Hello ",TokVar "man",TokText " and ",TokVar "woman",TokText "."]

>>> parse parseEverything "" "Hello ${man} and ${if(woman)} text here ${end}."
Right [TokText "Hello ",TokVar "man",TokText " and ",TokIf "woman",TokText " text here ",TokEnd,TokText "."]

>>> parse parseEverything "" "Hi ${for(people)} ${name}, ${end} everyone!"
Right [TokText "Hi ",TokFor "people",TokText " ",TokVar "name",TokText ", ",TokEnd,TokText " everyone!"]

>>> parse parseEverything "" "${realvar} $.get(javascript) $$ $$$ $} $( $45.50 $$escape $${escape2} wonderful life! ${truth}"
Right [TokVar "realvar",TokText " $.get(javascript) $$ $$$ $} $( $45.50 $$escape ",TokText "${",TokText "escape2} wonderful life! ",TokVar "truth"]

>>> parse parseEverything "" "<!--PREAMBLE  \n  foo: bar\ndo:\n  - re\n  -me\n  -->waffle house ${lyfe}"
Right [TokPreamble "  \n  foo: bar\ndo:\n  - re\n  -me\n  ",TokText "waffle house ",TokVar "lyfe"]

>>> parse parseEverything "" "YO ${foo} <!--PREAMBLE  \n  ${foo}: bar\ndo:\n  - re\n  -me\n  -->waffle house ${lyfe}"
Right [TokText "YO ",TokVar "foo",TokText " ",TokPreamble "  \n  ${foo}: bar\ndo:\n  - re\n  -me\n  ",TokText "waffle house ",TokVar "lyfe"]

This is a degenerate case that we will just allow (for now) to go sideways: >>> parse parseEverything "" "bthis ${var never closes/b ${realvar}" Right [TokText "bthis ",TokVar "var never closes/b ${realvar"]

Parse variables.

Parse preamble.

Parse the start of a PREAMBLE.

Parse partial commands.

>>> parse parsePartial "" "${partial(\"my/file/name.html\")}"
Right (TokPartial "my/file/name.html")

Parse escape sequence "$${"

>>> parse parseEscape "" "$${example}"
Right (TokText "${")

Parse boring, boring text.

>>> parse parseContent "" "hello ${ffwe} you!"
Right (TokText "hello ")

>>> parse parseContent "" "hello $.get() $ $( $$ you!"
Right (TokText "hello $.get() $ $( $$ you!")

Because of our first parser to grab a character that is not a $, we can't grab strings that start with a $, even if it's text. It's a bug, just deal with it for now. >>> isLeft $ parse parseContent "" "$$$ what" True

>>> isLeft $ parse parseContent "" "${name}!!"
True

Parse for loop declaration.

>>> parse parseFor "" "${for(posts)}"
Right (TokFor "posts")

>>> parse parseFor "" "${for(variable name with spaces technically allowed)}"
Right (TokFor "variable name with spaces technically allowed")

>>> isLeft $ parse parseFor "" "${for()}"
True

>>> isLeft $ parse parseFor "" "${for foo}"
True

Parse if directive.

General parse template functions.

Parse end keyword.

>>> parse parseEnd "" "${end}"
Right TokEnd

>>> isLeft $ parse parseEnd "" "${enddd}"
True

A hack to capture strings that "almost" are templates. I couldn't figure out another way.

many1Till p end will parse one or more p until @end.

From https://hackage.haskell.org/package/pandoc-1.10.0.4/docs/Text-Pandoc-Parsing.html

Find the preamble content from the given PNodes.

Returns True if the PNode is a PPreamble.

Gets the content of the PPreamble