May 15, 2025 · 55m · mad
Jeremy Howard on Building 5,000 AI Products with 14 People (Answer AI Deep-Dive)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this episode of The MAD Podcast, host Matt Turck interviews Jeremy Howard, co-founder of Answer.ai and Fast.ai, about building an Edison-inspired AI R&D lab that leverages open-source models and human-AI collaboration to ship thousands of products with a lean 14-person team. Howard critiques compute scaling hype, refutes near-term AGI claims, and showcases Answer.ai's innovative software stack including Solve-It, ShellSage, and FastHTML.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Matt holds 19.7% of the talking time here. How this is scored →
Matt as informed peer 3.5 Guest teaching 4.4 Guest disagreement 2.7 Matt pushing back 2.1
speaking balance: gold is Matt, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 8:00 Calling Out OpenAI's Compute Debacle
Jeremy forcefully criticizes OpenAI's spend-heavy culture, calling the GPT-4.5 release an inevitable debacle where costs far outweighed practical utility.
Hardest push from Matt ▶ 12:32 Challenging Diminishing Returns SkepticismMatt directly challenges Jeremy's skepticism about exponential progress, asking if he is dismissing genuine performance gains as mere psychological perception.
Biggest teaching moment ▶ 23:28 R&D Lab vs. Research Lab DistinctionsJeremy stops Matt to explicitly correct his premise, explaining why Answer.ai functions as an Edison-style R&D lab rather than a traditional academic research lab.
Matt holds his own ▶ 43:53 Mapping Answer.ai's Full Technical PortfolioMatt demonstrates deep domain preparation by reciting Answer.ai's obscure and varied software stack releases across encoders, search tools, and web frameworks.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Matt as informed peer | Guest teaching | Guest disagreement | Matt pushing back | Why |
|---|---|---|---|---|---|---|
| Reflections on ICLR Singapore and Academic vs. Practical AI | 4 | 4 | 3 | 2 | Matt guides the conversation across the model landscape, citing OpenAI's upcoming models, Phi, and Gemma. Jeremy provides practical realities from Answer.ai, dismissing vaporware press releases. | |
| The DeepSeek "Sputnik Moment" and Public Perception Shifts | 4 | 6 | 5 | 3 | Matt prompts Jeremy using Jeremy's own social media posts regarding OpenAI's business model. Jeremy forcefully critiques OpenAI's 4.5 compute spending and reframes public perception surrounding DeepSeek. | |
| Test-Time Compute, Inference Scaling, and Economic Realities | 5 | 6 | 4 | 4 | Matt presses Jeremy on whether raw model improvements contradict his view on diminishing exponential returns. Jeremy educates on inference compute limits and reframes AGI timeline hype as user interface perception tricks. | |
| Jeremy Howard's Early Path: Philosophy, McKinsey, and Kaggle | 2 | 3 | 2 | 1 | Matt prompts Jeremy on his academic background and transition into AI. Jeremy reflects on dropping out of university math and outperforming academics in early Kaggle competitions through self-taught practical experience. | |
| The Origins of Kaggle Inc. and the Creation of Fast.ai | 3 | 4 | 1 | 1 | Matt connects Kaggle to Fast.ai. Jeremy clarifies Fast.ai's underlying R&D cycle, using an analogy to restaurant research labs to show how teaching drives software development. | |
| Introduction to Answer.ai and Edison's R&D Lab Philosophy | 3 | 6 | 3 | 1 | When Matt introduces Answer.ai as an AI research lab, Jeremy immediately corrects the premise, emphasizing that it is an R&D lab modeled after Thomas Edison's Menlo Park laboratory. | |
| Structuring Answer.ai as a Public Benefit Corporation (PBC) | 4 | 3 | 1 | 2 | Matt demonstrates knowledge of Answer.ai's corporate structure and fundraising figures ($10M plus $8M). Jeremy outlines the Public Benefit Corporation charter and internal beta strategy. | |
| High Team Velocity and Evaluating Autonomous AI Agents (Devin) | 4 | 5 | 5 | 4 | Matt asks if future model improvements will fix autonomous agents like Devin. Jeremy strongly rejects fully autonomous handoffs, explaining why vibe-coded software fails unpredictable tasks. | |
| Solve-It: Reimagining AI Workflows via Dialogue Engineering | 3 | 4 | 2 | 2 | Matt asks if Solve-It is strictly a coding tool. Jeremy clarifies that it spans non-coding domains like book writing and business operations through dialogue engineering. | |
| Overcoming LLM Limitations through Agentic Search and R&D Loops | 3 | 4 | 3 | 2 | Matt asks how Solve-It circumvents common LLM issues like hallucinations. Jeremy details grounding mechanisms and notes he doesn't face common user complaints due to his human-in-the-loop workflow. | |
| ShellSage: Bringing Context-Aware AI into the Command-Line Terminal | 2 | 4 | 1 | 1 | Matt transitions to secondary tools. Jeremy details how ShellSage leverages TMUX to feed full terminal context into an LLM session. | |
| Building the FastHTML, Monster UI, and Plash Developer Stack | 5 | 6 | 3 | 2 | Matt lists Answer.ai's tool suite but mispronounces ColBERT, which Jeremy playfully corrects. Jeremy then details Python M-expressions and web stack efficiencies behind FastHTML. | |
| Defining the AI Substrate and Team Architecture | 4 | 4 | 3 | 3 | Matt inquires about product selection criteria among thousands of options. Jeremy explains the underlying substrate concept and non-traditional management systems built into Discord. | |
| Five-Year Outlook and Product Strategy | 3 | 3 | 2 | 1 | Matt asks about long-term team scaling. Jeremy rejects traditional headcount expansion, outlining a 5-year focus on building commercial consumer applications rather than raw model infrastructure. |
Statements from this episode (24)
Howard: Answer.ai has completely diverged from mainstream academic AI research
“Quickly realized, actually, we've gotten so far off the, you know, academic normal research check now that No one cares about anything we care about on the whole and vice versa.”
Jeremy Howard: Two of Answer.ai's three production models are open source
“Two of the three models that we rely on in our day-to-day production stuff are open source.”
Howard: Closed US AI ecosystems will cause China to move faster
“Whereas, oddly, the US companies that used to lead the way have all drawn up the drawbridges, and that's gonna cause China to keep moving faster, because when you're in that more, both collaborative and competitive environment, you just Go way ahead, as we've …”
Jeremy Howard: OpenAI's Deep Research is still the best tool available
“Their deep research tool is still the best.”
Howard: There was no technological breakthrough 'DeepSeek moment'
“For me, there was no technology DeepSeek moment.”
Howard: OpenAI compute spending grows exponentially while model utility scales logarithmically
“They kept on kind of exponentially increasing the amount they were spending on their models, whilst the Return, you know, the kind of utility of those models was only increasing logarithmically, and you kind of very quickly hit this point where it's like, oh, …”
Howard: OpenAI is shutting down GPT-4.5
“I think they're shutting down that product or they've shut down that product, if I understand correctly.”
Howard: Test-time compute scaling will hit diminishing returns within two years
“It's a thing where you get most of the juice out of it in the first year or two, so we're still in that. Phase at the moment, and we'll start to hit the curve off point pretty soon. Just like we did for training.”
Howard: No more evidence for near-term ASI today than 15 years ago
“I don't think we have any more evidence that ASI might be close now than we did 15 years ago, 15 years before that.”
Howard: AI growth will continue tailing off as low-hanging fruit disappears
“The amount of people and money being put into AI, it's going to keep going up, but again, the low hanging fruits being done now. So yeah, I think we'll continue to see the tailing off of growth.”
Howard: Non-autoregressive AI architectures will deliver major performance gains
“But I think it probably will work. And I think that'll probably be a significant jump in performance when you can sketch out the entirety of the solution first, and then fill in the blocks and gradually increasing levels of specificity.”
Jeremy Howard: Academic approaches to predictive modeling are far less successful than practical experience
“Much to my surprise, the academic approach is, was way less successful.”
Howard: Answer.ai is an Edison-inspired R&D lab, not a research lab
“So it's definitely not a research lab. This might sound minor, but it's an R and D lab, but they feel and look extremely different as was Edison's Menlo Park lab was an R and D lab.”
Howard: Traditional AI labs favor paper citations over bold innovation
“It's very different to a research lab, which is like, hey, let's try and get lots of citations on a paper in a field that's so highly recognizable to its peers that they recognize it as being something they want to appear at their next conference, you know, wh…”
Answer.ai has 14 employees but aims to operate with only 12
“Yeah, maybe 14 now. 14 We're kind of aiming to be around 12. There was a couple of people we found we couldn't say no to.”
Jeremy Howard: Humans and AI produce better results collaborating in shared environments
“Humans and computers work better together than separately, and then there are various things that kind of come out of that, and one key one, for example, is that the human and the computer should be able to see all of each other's work and be operating in the …”
Howard: Autonomous AI coding agent Devin produces low-quality, useless software
“Devon's an agent you know, a bunch of tools, tool calls, and tied together with an OpenAI model, if I understand correctly. Yeah, and it actually ended up definitely supporting our thesis, which is, there are so many places we wished we could have got involved…”
Jeremy Howard: Autonomous AI agents are only useful for small, simple tasks
“There are certainly situations where, like, Scripts, spreadsheets, VBA, access databases are useful, you know, but nobody like builds them as the components of their technology, you know, strategy. They're the things you use for little tools. It's small, well-…”
Jeremy Howard sees no reason to believe Artificial Superintelligence will occur
“I have no particular reason to believe that will happen, and for every point until we get there, if we ever get there, by definition, there's going to be humans interacting with AI, and so that's what I care about, is how do we do that In an optimal way.”
Howard: Answer.ai team uses AI at a more advanced level than anyone else
“Definitely I feel like we are more, I can tell we're more advanced users of AI than anybody else, because people can keep complaining about all the problems with AI, and I'm always like, I don't have any of those problems, you know.”
Howard: Answer.ai aims to launch 5,000 successful products with 14 people
“If we're going to have 12 to 14 people create five to 10,000 extremely commercially successful products, we're going to have to be extremely efficient, you know, at every level.”