Arabic
1 statements across 1 episodes · 0 bullish · 0 bearish · 1 people on the record · first statement Sep 7, 2023 by Andrew Feldman · across every show →
Everything said about Arabic, oldest first
Sep 7, 2023
Feldman: Poor tokenization schemes heavily bias foundation models toward English
“One of the things we learned was that it's really important taking into account
The language that has fewer tokens, especially, you know, Arabic, Hebrew, Hindi, their characters sometimes require two bits rather than one bit, and so the tokenization scheme has…”