html-parse: A high-performance HTML tokenizer

This package provides a fast and reasonably robust HTML5 tokenizer built upon the attoparsec library. The parsing strategy is based upon the HTML5 parsing specification with few deviations.

For instance,

>>> parseTokens "<div><h1 class=widget>Hello World</h1><br/>"
[TagOpen "div" [],
TagOpen "h1" [Attr "class" "widget"],
ContentText "Hello World",
TagClose "h1",
TagSelfClose "br" []]

The package targets similar use-cases to the venerable tagsoup library, but is significantly more efficient, achieving parsing speeds of over 80 megabytes per second on modern hardware and typical web documents. Here are some typical performance numbers taken from parsing a Wikipedia article of moderate length:

benchmarking Forced/tagsoup fast Text
time                 186.1 ms   (175.3 ms .. 194.6 ms)
0.999 R²   (0.995 R² .. 1.000 R²)
mean                 191.7 ms   (188.9 ms .. 198.3 ms)
std dev              5.053 ms   (1.092 ms .. 6.809 ms)
variance introduced by outliers: 14% (moderately inflated)

benchmarking Forced/tagsoup normal Text
time                 189.7 ms   (182.8 ms .. 197.7 ms)
0.999 R²   (0.998 R² .. 1.000 R²)
mean                 196.5 ms   (193.1 ms .. 202.1 ms)
std dev              5.481 ms   (2.141 ms .. 7.383 ms)
variance introduced by outliers: 14% (moderately inflated)

benchmarking Forced/html-parser
time                 15.81 ms   (15.75 ms .. 15.89 ms)
1.000 R²   (1.000 R² .. 1.000 R²)
mean                 15.72 ms   (15.66 ms .. 15.77 ms)
std dev              140.9 μs   (113.6 μs .. 174.5 μs)

• Text
  • HTML
    • Text.HTML.Parser
    • Text.HTML.Tree