Safe Haskell | None |
---|---|
Language | Haskell98 |
Analyzer for index data.
Creates raw index data by splitting and normalizing the ApiDocument
index data as defined in
the schema.
- toDocAndWords :: DocumentWrapper e => Schema -> ApiDocument -> (e, Score, Words)
- toDocAndWords' :: Schema -> ApiDocument -> (Document, Score, Words)
- normalize :: CNormalizer -> Text -> Text
- scanTextRE :: RegEx -> Text -> [Word]
Documentation
toDocAndWords :: DocumentWrapper e => Schema -> ApiDocument -> (e, Score, Words) Source
Extracts the Document
(DocumentWrapper
) and raw index data from an ApiDocument
in
compliance with the schema.
Note: Contexts mentioned in the ApiDocument
need to exist.
toDocAndWords' :: Schema -> ApiDocument -> (Document, Score, Words) Source
Extracts the Document
and raw index data from an ApiDocument
in compliance with the schema.
Note: Contexts mentioned in the ApiDoc need to exist.
normalize :: CNormalizer -> Text -> Text Source
Normalization function.
scanTextRE :: RegEx -> Text -> [Word] Source
Tokenize a text with a regular expression for words.
scanTextRE "[^ \t\n\r]*" == Data.Text.words