Package opennlp.tools.tokenize

# Class DictionaryDetokenizer

java.lang.Object
opennlp.tools.tokenize.DictionaryDetokenizer

All Implemented Interfaces:
  `Detokenizer`

public class DictionaryDetokenizer extends Object implements Detokenizer
A rule based detokenizer. Simple rules which indicate in which direction a token should be moved are looked up in a `dictionary`.

See Also:
  - `Detokenizer`
  - `DetokenizationDictionary`

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from interface opennlp.tools.tokenize.Detokenizer

  `Detokenizer.DetokenizationOperation`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `DictionaryDetokenizer(DetokenizationDictionary dict)`
  Initializes a `DictionaryDetokenizer` instance.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `Detokenizer.DetokenizationOperation[]`
  `detokenize(String[] tokens)`
  Detokenizes the collection of tokens.
  `String`
  `detokenize(String[] tokens, String splitMarker)`
  Detokenizes the input `tokens` into a String.

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### DictionaryDetokenizer

    public DictionaryDetokenizer(DetokenizationDictionary dict)
    Initializes a `DictionaryDetokenizer` instance.
    Parameters:
      `dict` - The `DetokenizationDictionary` to be used.

- ## Method Details

  - ### detokenize

    public Detokenizer.DetokenizationOperation[] detokenize(String[] tokens)
    Description copied from interface: `Detokenizer`
    Detokenizes the collection of tokens.
    Specified by:
      `detokenize` in interface `Detokenizer`
    Parameters:
      `tokens` - The elements which should be detokenized.
    Returns:
      The `merge operations` to handle given `tokens`.

  - ### detokenize

    public String detokenize(String[] tokens, String splitMarker)
    Description copied from interface: `Detokenizer`
    Detokenizes the input `tokens` into a String. Tokens which are connected without a `whitespace` character in between can be separated by a given `splitMarker`.
    Specified by:
      `detokenize` in interface `Detokenizer`
    Parameters:
      `tokens` - The elements which should be concatenated.
      `splitMarker` - The split marker or `null`.
    Returns:
      The concatenated tokens as a single string.