Skip to main content
Version: 0.2.4

Module: text/WordpieceTokenizer

Classes​

Type aliases​

WordPieceTokenizerConfig​

Ƭ WordPieceTokenizerConfig: Object

Type declaration​

NameType
lowercase?boolean
neverSplit?string[]
unknownToken?string
vocabstring