Skip to main content
Version: 0.2.3

Module: text/WordpieceTokenizer

Classes

Type aliases

WordPieceTokenizerConfig

Ƭ WordPieceTokenizerConfig: Object

Type declaration

NameType
lowercase?boolean
neverSplit?string[]
unknownToken?string
vocabstring