trie tokenizer
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Aug 31, 2023 by Eugene Cheah · across every show →
Everything said about trie tokenizer, oldest first
Aug 31, 2023 positive
RWKV Uses Trie Tokenizer Without Space Delimiters for CJK Languages
“Instead of using like this token pairs well with this and should be paired with that we just made it a trial list. So So basically, try the data structure. Yeah. So we just find the longest matching string in that matching string that we have trained inside ou…”