nerf-0.3.0: Nerf, the named entity recognition tool based on linear-chain CRFs

Safe HaskellNone

NLP.Nerf.Tokenize

Contents

Description

The module implements the tokenization used within Nerf and some other tokenization-related stuff.

Synopsis

Tokenization

tokenize :: String -> [String]Source

Tokenize sentence using the default tokenizer.

Synchronization

class Word a Source

A class of objects with size.

Instances

moveNEs :: Word b => NeForest a b -> [b] -> NeForest a bSource

Synchronize named entities with tokenization represented by the second function argument. Of course, both arguments should relate to the same sentence.