Everything Alex Ratner said on any show that made the record, most notable first. Each card names its show and opens the statement there.
Alex Ratner: Traditional model-centric AI development is dying off
“Tweaking the model, finding a fancier algorithm, all this kind of traditional model-centric development that, frankly, we still mostly teach in data science one-on-one courses as what data science is, is effectively dying off.”
Alex Ratner: Enterprise data is AI's only durable moat
“Enterprise data knowledge is the durable moat in AI, I'd argue the only durable one”
Ratner: $200 of ChatGPT calls can clone closed models into open ones
“If you take a couple hundred bucks of API calls to, say, ChatGPT, and you graph that onto a model that is substantially smaller, say a seven billion parameter model like LLAMA, or now increasingly fully open for commercial use ones, like Red Pajama is one that…”
Ratner: Private data models will exceed closed models in specialized tasks
“Closed source models, like a GPT-IV, five, six, seven, whatever comes, are going to be very hard to match in terms of generalist capability for, say, consumer use cases that are reflected in the web data they're trained on and the flywheels that get powered by…”
Ratner: Distillation creates specialized models 100,000x smaller with higher accuracy
“It shows that you can take these massive kind of generalist foundation models, And not only can you tune them using that kind of programmatic labeling to be more accurate on a given task that you need to be really accurate on, but you can then distill down ver…”
Ratner: Base models will open-source while production uses specialized models
“There are going to be these base models. They're probably going to be increasingly open source. And then the reality of what actually ships in production is going to be a whole kind of family tree of smaller specialized models that are tuned and adapted via da…”
Ratner: DataComp benchmark beat OpenAI models purely through data curation
“Just by cleaning, curating, sampling, filtering the data, we get a new state of the art score at compute parity, beating open AI models and others.”
Ratner: Foundation models are incomplete solutions for real-world enterprise AI
“In most real-world use cases and we'll get into how I define that in a bit, there's still a last mile to be traversed. So it's not that they're overhyped, per se, they're just not the complete solution.”
Ratner: Foundation models will require customization on enterprise-specific data
“Most of these foundation models, like a GPT-IV, five, six, seven, are going to need to be customized on your specific data and knowledge and workloads.”
Ratner: BloombergGPT achieves only low 60s accuracy on specialized financial tasks
“If you actually open the first page of the Bloomberg GPT paper, the FinServe tasks that it does better on by using private financial data are still getting low sixties in terms of accuracy.”
MADLEDGER