GPT-III

“Every year or two while starting Replit, I would go look at the state of the art on ML on code and nothing ever really worked all that well up until GPT-II. And you can like take GPT-II and fine tune it and like try to make it write code and was like kind of o…”

“GPT-III was trained on 10,000 V-one-hundred GPUs in an interconnected cluster in Azure for about 14.6 days.”

“I think in the protein, protein models, we are past GPT-III. when it comes to single cell models and virtual cell models, yeah, I think GPT-I to two right now. I think we're closer to GPT-I than two.”

“Like, with GPT-III, you couldn't scale test time compute. Like, if you gave it a budget of ten million dollars and said, okay, well, let's see what GPT-III can do, it really can't do that much, more than what you could do with, like, 10 dollars or one dollar.”