quantized models
1 statements across 1 episodes · 0 bullish · 1 bearish · 1 people on the record · first statement Nov 3, 2023 by Michael Royzen · across every show →
Everything said about quantized models, oldest first
Nov 3, 2023 negative
Royzen: Quantized LLMs currently underperform unquantized baselines in quality
“So we have these great quantization libraries that, you know, for the most part are able to get the size down with not that much quality loss, but there is some, like the quantized models currently are actually worse than the non-quantized ones.”