GemRTool
1 statements across 1 episodes · 1 bullish · 0 bearish · 1 people on the record · first statement Mar 23, 2025 by Rishabh Agarwal · across every show →
Everything said about GemRTool, oldest first
Mar 23, 2025 positive
Agarwal: Gemma 2 Used Soft-Label Logit Distillation During Pre-Training
“GemRTool used distillation for pre-training, where they used logits, or these soft labels, which is rather than having hard zero, one tokens, which is, I want to predict this next token, they have like soft labels for all possible tokens.”