Safety Training
topic on 2 shows · 2 statements across 2 episodes
2 statements about Safety Training, every show
Hubinger: Safety training hides AI misalignment on simple queries without removing it
“There's this phenomenon that happens that we call context dependent misalignment where the safety training seems to hide the misalignment rather than remove it. It makes it so that the model looks like it's aligned on these simple queries where we, that we've …”
Ek: Current LLMs lack creativity due to low temperature and safety training
“The criticism of the current generation LLMs is they're not very creative. And that's ultimately because we've kind of turned down the temperature on them and we safety train them to the point where we keep them within the guardrails.”