CARBS, every mention
8 scenes · ← back to CARBS
tap a year for its mentions
every year anyone Kanjun Qiu 4Josh Albrecht 3
Verbatim, from the transcripts: the passages where CARBS comes up
State of the Art: Training 70B LLMs on 10,000 H100 clusters
- ▶ 8:17 Josh Albrecht And then the third thing is carbs, our hyperparameter, our cost aware hyperparameter optimizer, which was especially helpful for being able to experiment in much smaller scales and then scale those experiments up to the much larger scale…
- ▶ 46:22 unnamed speaker Um, I wanted to go into carbs, um, since like, that's like kind of the next layer of the stack.
- ▶ 46:27 unnamed speaker Um, we talked about carbs in the first episode with Kanjin because you've actually had a blog post about it like a couple of years ago. 6 times in the scene
- ▶ 51:59 Josh Albrecht And so carbs actually control what is the mix of data that we want to see, like how much code, you know, how much internet text, et cetera, uh, in order to figure out what is the best optimal mix of data. 2 times in the scene
- ▶ 55:39 unnamed speaker Doesn't emergence throw a spanner node theory of carbs? 3 times in the scene
- ▶ 57:56 unnamed speaker Um, so carbs, we already mentioned, um, you know, leans heavily on, uh, the sort of end evals that we would typically eval LMs on, except that you had to make your own.
Why AI Agents Don't Work (yet) - with Kanjun Qiu of Imbue
- ▶ 16:39 Kanjun Qiu CARBS, our hyperparameter optimizer, came from Abe trying to automate his own research process, uh, doing hyperparameter optimization, and he actually pulled some ideas from plasma physics, he's a plasma physicist, to make the local search…
- ▶ 57:17 Kanjun Qiu Um, like the curiosity led us to, uh, build carbs, for example, uh, this hyperparameter optimizer. 3 times in the scene