why aren't all 12 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 0 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Insight
Kanjun Qiu: AI emergent capabilities are artifacts of evaluation metric design
“If your metric is smooth you actually see slow performance improvement over time, and if your metric is relatively discrete or not smooth, that's where you see the emergence, and it's actually more about the evaluation metric Than about the emergence of the ca…”
Opinion
Qiu: AI is currently in an un-reliable analog computer phase
“I think today, you know, we're kind of in the analog computer phase of AI, where You know, ah, these systems, they output things. Errors compound. They're not very reliable.”
Assertion Supported
Kanjun Qiu: Language models trained without code perform poorly at reasoning
“Some companies have tried training language models with no code because they're like, oh, the product we're building, we don't need code. You know, it's a therapist or it's a question answering thing and we don't need code. So we should take code out of our tr…”
Insight
Qiu: Psychological trauma can be conceptualized as computational overfitting
“Trauma is overfitting and actually a good way to overcome trauma. If we look at a lot of therapy It's like by giving more data to me and helping me access the overfit parts and then give them more data.”
Insight
Kanjun Qiu: Self-supervised learning mirrors how humans naturally learn
“Self-supervised learning is interesting because it's how humans learn, you know, most of the time we go around We like Google stuff. We learn on our own. We're not giving supervised labels all the time.”
Disclosure
Kanjun Qiu: 95% of Imbue's work is engineering, not research
“So the company is mostly engineering. You know, I think something people, when people think about AI, they're like, oh, it's research, but really, 95% of the work that we all do is engineering work.”
Insight
Qiu: Current AI models are book smart but lack street smarts
“The way to think about today's models is they, they're like book smart, but not street smart. They're like a person who's read the whole internet, but they've never done anything in the world.”
Assertion Supported
Kanjun Qiu: Imbue operates a cluster of 10,000 Nvidia H100 GPUs
“And then NVIDIA is also part of the round because we have a very large GPU cluster of 10,000 H 100, and it's very helpful to have NVIDIA as part of that.”
Disclosure
Kanjun Qiu paid for MIT by designing high-frequency trading algorithms
“I paid for MIT by trading high-frequency trading algorithms, so by designing high-frequency trading algorithms”
Disclosure
Qiu: Imbue pre-trains 100B+ parameter models from scratch
“We pre-train our own large model. So, you know, very large hundred billion parameter plus models from scratch with our own data.”
Assertion Not publicly verifiable
Imbue used plasma physics principles to build its CARBS search algorithm
“Like the reason we have carbs is because Abe was a plasma physicist and he applied some ideas from plasma physics, physics to make this local search algorithm work well.”
Assertion Supported
GPT-3 lead authors Tom Brown and Ben Mann were Qiu's housemates
“And out of that house our housemates, Tom Brown and Ben Mann are the first two authors on GBT-III.”