AI Architecture
topic on 7 shows · 11 statements across 11 episodes
the Y Combinator Startup Podcast 
Cheeky Pint 
We Live to Build 
the Neon Show 
No Priors 
Big Technology 
All-In
11 statements about AI Architecture, every show
May: AI architecture will mirror software transition to microservices
“Most companies start with a monolithic code base, everything in one repo and one giant thing. And then they start to parse off the pieces that need to be optimized and make them microservices. And I mean, it's, that's how businesses are going to go with AI as …”
Chollet: AI Architectures Requiring Human Engineers to Scale Will Fail
“If you're working on something, but the only way to increase the capabilities of the system Is to have human engineers and researchers spend time on it. It will not work because even if the idea is very clever and very elegant and works really well, capabiliti…”
Dolgov: Autonomous AI relies more on metrics and data than architecture
“And of course, you know, people like to talk about architectures, but architecture, you know, is important, but really a lot of it comes down primarily to your metrics, to your evaluation mechanisms, to, you know, all of the training recipes, and of course, ne…”
Guo: Reasoning is the biggest AI architecture shift since the transformer
“Reasoning is the biggest paradigm shift in AI architecture since the transformer.”
LeCun: Practical world-model AI architectures will take three to five years
“It's not going to happen within the next three years, but it may happen with, you know, between three to five years, something like that”
DeepMind CTO: AI algorithms and architecture matter just as much as scale
“When we are thinking about our architectures,
Like the architectural elements, the algorithms that we put in there that come, that make up the model, right?
They are as important as the scale.”
LeCun: AI will eventually ask its own questions, taking considerable time
“You know, is, are we going to have eventually AI architectures, AI systems that are capable of not just answering questions that are already there, but solving, giving new solutions to problems that we specify? The answer is yes, eventually. Not with current L…”
Sergey Brin: AI development is trending toward unified, shared architectures
“But I do think the trend is to have a more unified model. I don't know if I'd call it a god model. But to have certainly sort of shared architectures and ultimately even shared models.”
Central routing across specialized AI subsystems will unlock the field
“I think the key element that is going to really unlock this field Is actually going to be the router in the middle of a series of different systems which are specialized, some of which don't even look like AI at all. They might just be traditional pieces of so…”
Goyal: Bulk of Software Incumbents Will Fail to Adapt to AI
“Some incumbents will get their act together. Bulk of the incumbents will not get their act together.”
LeCun: World models, not more data, will solve AI common sense
“So I think this is not going to be solved in my opinion by you know, more tweaks on the architectures and more training data and things like this. I mean, it may be mitigated, but I think it's not going to be fixed by that. I think it's going to be fixed by sy…”