Skip to main content
Version: 0.2.0

Module: text/WordpieceTokenizer

Classes​

Type aliases​

WordPieceTokenizerConfig​

Ƭ WordPieceTokenizerConfig: Object

Type declaration​

NameType
lowercase?boolean
neverSplit?string[]
unknownToken?string
vocabstring

Defined in​

text/WordpieceTokenizer.ts:12