Selector defines a selection of an HTML DOM tree to be operated on by a web scraper. The selection includes the opening tag that matches the selection, all of the inner tags, and the corresponding closing tag.

An AttributePredicate is a method that takes a Attribute and returns a Bool indicating if the given attribute matches a predicate.

The AttributeName type can be used when creating Selectors to specify the name of an attribute of a tag.

The TagName type is used when creating a Selector to specify the name of a tag.

A selector which will match all tags

The // operator creates an Selector by nesting one Selector in another. For example, "div" // "a" will create a Selector that matches anchor tags that are nested arbitrarily deep within a div tag.

The @: operator creates a Selector by combining a TagName with a list of AttributePredicates.

The @= operator creates an AttributePredicate that will match attributes with the given name and value.

If you are attempting to match a specific class of a tag with potentially multiple classes, you should use the hasClass utility function.

The @=~ operator creates an AttributePredicate that will match attributes with the given name and whose value matches the given regular expression.

The classes of a tag are defined in HTML as a space separated list given by the class attribute. The hasClass function will match a class attribute if the given class appears anywhere in the space separated list of classes.

Negates an AttributePredicate.

The match function allows for the creation of arbitrary AttributePredicates. The argument is a function that takes the attribute key followed by the attribute value and returns a boolean indicating if the attribute satisfies the predicate.

A value of Scraper a defines a web scraper that is capable of consuming a list of Tags and optionally producing a value of type a.

The attr function takes an attribute name and a selector and returns the value of the attribute of the given name for the first opening tag that matches the given selector.

This function will match only the opening tag matching the selector, to match every tag, use attrs.

The attrs function takes an attribute name and a selector and returns the value of the attribute of the given name for every opening tag that matches the given selector.

The html function takes a selector and returns the html string from the set of tags described by the given selector.

This function will match only the first set of tags matching the selector, to match every set of tags, use htmls.

The htmls function takes a selector and returns the html string from every set of tags matching the given selector.

The innerHTML function takes a selector and returns the inner html string from the set of tags described by the given selector. Inner html here meaning the html within but not including the selected tags.

This function will match only the first set of tags matching the selector, to match every set of tags, use innerHTMLs.

The innerHTMLs function takes a selector and returns the inner html string from every set of tags matching the given selector.

The text function takes a selector and returns the inner text from the set of tags described by the given selector.

This function will match only the first set of tags matching the selector, to match every set of tags, use texts.

The texts function takes a selector and returns the inner text from every set of tags matching the given selector.

The chroot function takes a selector and an inner scraper and executes the inner scraper as if it were scraping a document that consists solely of the tags corresponding to the selector.

This function will match only the first set of tags matching the selector, to match every set of tags, use chroots.

The chroots function takes a selector and an inner scraper and executes the inner scraper as if it were scraping a document that consists solely of the tags corresponding to the selector. The inner scraper is executed for each set of tags matching the given selector.

The position function is intended to be used within the do-block of a chroots call. Within the do-block position will return the index of the current sub-tree within the list of all sub-trees matched by the selector passed to chroots.

For example, consider the following HTML:

<article>
 <p> First paragraph. </p>
 <p> Second paragraph. </p>
 <p> Third paragraph. </p>
</article>

The position function can be used to determine the index of each <p> tag within the article tag by doing the following.

chroots "article" // "p" $ do
  index   <- position
  content <- text "p"
  return (index, content)

Which will evaluate to the list:

[
  (0, "First paragraph.")
, (1, "Second paragraph.")
, (2, "Third paragraph.")
]

The scrape function executes a Scraper on a list of Tags and produces an optional value.

The scrapeStringLike function parses a StringLike value into a list of tags and executes a Scraper on it.