nerf-0.3.0: Nerf, the named entity recognition tool based on linear-chain CRFs
NLP.Nerf.Tokenize
Contents
Description
The module implements the tokenization used within Nerf and some other tokenization-related stuff.
Synopsis
tokenize :: String -> [String]Source
Tokenize sentence using the default tokenizer.
class Word a Source
A class of objects with size.
Instances
moveNEs :: Word b => NeForest a b -> [b] -> NeForest a bSource
Synchronize named entities with tokenization represented by the second function argument. Of course, both arguments should relate to the same sentence.