Gil: Longer context windows will prove more important than expected for biology
Elad Gil · No Priors Ep. 54 | With Sarah Guo & Elad Gil · Mar 7, 2024 · at 0:46
Investor Elad Gil discusses the practical implications of large model context windows, referencing Google Gemini's million-token capabilities.
0:00 / 0:30exact quote · 30.7s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I think for certain application areas, like biology, longer context windows actually seem to be quite important. And so, for example, if you're doing a protein folding model and you have a short context window, you're often actually not encapsulating much of the protein, right? The average protein is, I think, something like 300 amino acids long, at least in the human genome, but there are things that are dramatically larger than that. And so you just can't capture it in some of the context when it was being used for biological models. And so I do think this is going to be one of those areas that will end up being more important than people think, at least in the short run.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Elad Gil
Gil: California is discussing an exit tax on departing residents by 2028
“And my sense is in 28, there's increasing talk about also trying to add a exit tax in California. So if you actually try and leave, they'll, they'll try and take a big chunk as sort of a penalty for that.”
Gil: Chinese Open-Source Subsidies Indirectly Subsidize US Enterprise AI Adopters
“At least for now, it looks like effectively the Chinese government is subsidizing at least a large subset of these models, and that subsidy or surplus is effectively just being passed on to US enterprises We're adopting these models. In other words, it's a way…”
Gil: Closed-Source AI Labs Use Existential Risk Narratives for Regulatory Capture
“It's a very traditional form of regulatory capture. If you have a big pharma company, they work with the government to prevent other entrants into the industry. And this is exactly what it felt like is at least partially happening. In the AI world. Now that ma…”
Gil: Many big tech companies could shrink by 70% and improve
“If you look at the big tech companies, for example, right now, many, not all of them, but many of them could probably shrink by 70% and be more effective, right?”
Gil: Government, academia, and big tech have effectively functioned as UBI
“The degree to which we effectively had different forms of UBI or universal basic income in different sectors of the economy. Government is a clear example where there's enormous waste, fraud, grift, et cetera, happening. Parts of academia, if you just look at …”
Elad Gil argues mainstream use of artificial wombs would harm society
“There's advances in science where, you know, you can now grow I can't if they use lamb or what, you know, up to some, you know, number of weeks of gestation, right? In a literal kind of membrane that's fed in nutrients and stuff. And if you'd asked me 10 years…”