word
tokenization
toh-ken-ih-ZAY-shun
toʊkənɪˈzeɪʃənListen
Hear it in a clear American voice. Slow it down to catch each syllable.
Syllable breakdown
tokenization
Dictionary meaning
breaking text into smaller units (tokens) for processing by language models
People also look up
Learn one new pronunciation a day
Get a word, name or idiom like “tokenization” in your inbox — daily or weekly, free. Unsubscribe any time.
Something wrong? Suggest a correction