Quantized Models
topic on 1 show · 1 statements across 1 episodes
1 statements about Quantized Models, every show
Royzen: Quantized LLMs currently underperform unquantized baselines in quality
“So we have these great quantization libraries that, you know, for the most part are able to get the size down with not that much quality loss, but there is some, like the quantized models currently are actually worse than the non-quantized ones.”