Beyang Liu: Simpler baseline systems often match or beat fancy AI models
Beyang Liu · No Priors Ep. 47 | With Sourcegraph CTO Beyang Liu · Jan 18, 2024 · at 30:33
Sourcegraph CTO Beyang Liu discusses AI system design philosophy and why engineering teams should build simple baselines before training complex neural network models.
0:00 / 0:50exact quote · 50.4s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Doing the simple thing, it establishes a baseline. Like oftentimes you'll find that like the doing the fancier thing is often sexier. And it's certainly these days it's like trendier, right? Cause you can kind of claim the mantle of like, ah, you know, I made a, I made my own LLM beyond LLM and it, you know, I trained it on my own data. Now I'm, I have, you know, AI or ML street cred because I did something at the model layer, but the lesson I took away from my research days was really the importance of establishing a baseline because oftentimes if you do the fancy thing first you might have something that looks like a good result because it, you know, it's gonna work to some degree but then someone else might come along and do a much dumber simpler thing, cheaper, and one that can be improved more iteratively, And it's going to work as well or better than your solution.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Beyang Liu
Beyang Liu: Keyword Search Solves Over 90% of RAG Retrieval
“I think if you talk to a lot of RAG practitioners you'll find that the kind of, like, dirty secret is that keyword search can probably get you more than 90% of the way there.”
Beyang Liu: RAG remains necessary for context even with fine-tuned models
“I think you're still going to want to do RAG anyways. Like, even if you have fine tuned models in the mix, RAG is still sort of this, like, last mile data or context.”
Liu: Single Developers Will Build Apps That Currently Require Large Teams
“If your goal is to build software as it exists today, then yes, I think in the future, a single human will be able to build applications that today require large numbers of people to coordinate.”
Liu: Software Teams Will Remain Large as Scope and Ambition Expand
“And so I still think we will have large teams of software developers in the future. They will just You know, each individual will be responsible for far more feature scope than they are today, and the system as a whole will be more sophisticated and more power…”
Beyang Liu: CS fundamentals will grow in importance in AI era
“My, maybe a bit contrarian hot take here is that CS Fundamentals if anything, are gonna grow in importance. You know, the stuff you learn in the coding bootcamp, maybe that gets, you know, automated away, but the fundamentals of knowing, you know, which data s…”
Beyang Liu: Statistical learning and convex optimization will re-emerge in AI
“I'm still waiting for the statistical learning and maybe some of the convex optimization stuff to reemerge. I wouldn't count it entirely out yet. I feel like the pendulum always swings back the other way. It's swung away from statistical learning and convex op…”