# Uses of Interface
opennlp.tools.tokenize.TokenContextGenerator

Packages that use TokenContextGenerator
Package
Description
opennlp.tools.tokenize
Contains classes related to finding token or words in a string.
opennlp.tools.tokenize.lang

- ## Uses of TokenContextGenerator in opennlp.tools.tokenize

  Classes in opennlp.tools.tokenize that implement TokenContextGenerator
  Modifier and Type
  Class
  Description
  `class`
  `DefaultTokenContextGenerator`
  A default `TokenContextGenerator` which produces events for maxent decisions for tokenization.
  Methods in opennlp.tools.tokenize that return TokenContextGenerator
  Modifier and Type
  Method
  Description
  `TokenContextGenerator`
  TokenizerFactory.`getContextGenerator()`
  Constructors in opennlp.tools.tokenize with parameters of type TokenContextGenerator
  Modifier
  Constructor
  Description
  ``
  `TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg)`
  Initializes a new event stream based on the data stream using a `TokenContextGenerator`.
  ``
  `TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg)`
  Initializes a new event stream based on the data stream using a `TokenContextGenerator`.

- ## Uses of TokenContextGenerator in opennlp.tools.tokenize.lang

  Methods in opennlp.tools.tokenize.lang that return TokenContextGenerator
  Modifier and Type
  Method
  Description
  `TokenContextGenerator`
  Factory.`createTokenContextGenerator(String languageCode, Set<String> abbreviations)`
  Initializes a customized `TokenContextGenerator` via a set of `abbreviations`.