The Ledger, every show
Every statement that passed quotation and attribution checks, across all 44 shows. Pick shows below, then mix any filter with any other.
shows 
every show 44 of 44
Memory tuning eliminates hallucinations and enables near-perfect task performance
“Been able with memory tuning, which is what I've been working on to remove those hallucinations, to remove that and actually get these models from, you know, not necessarily being general for everything. And instead of being pretty good at everything, but perf…”
The enterprise GPU shortage has eased at the company level
“Today, actually, I'm seeing the GPU shortage go away at the level, at the company level, meaning companies are able to procure enough compute enough is a strong word, but they're able to procure compute at some level to work with, to fine tune and run heavy in…”
In mid-2023, multi-billion dollar companies could not obtain AWS GPU nodes
“Last year was, at this time, was absolutely insane. That's why we threw up our own cloud, because there was just like, large companies with multi-billion revenue numbers could not get a node from AWS, despite their accounts being tens of millions or hundreds o…”
Re-engineering LLM decoders can guarantee absolute schema accuracy for structured outputs
“So that's something else we offer through our inference service to actually make it a hundred percent by re-engineering the decoder of any LLM.”
Future AI models will deliver 100B parameter intelligence at 1B speeds
“I even think there's a future where these models can be a hundred billion parameters, but at, you know, have that intelligence of a hundred billion parameters, but then have the speed, latency, and cost of something that's still one billion or seven billion pa…”
Combining MoE and LoRA will eliminate big versus small model trade-offs
“And I do think that's the future so we can get something that is incredibly smart, incredibly huge, but with the latency cost and speed of something, something tiny. So no more big model versus small model paradigm. It's potentially one in the same.”
Adding sequential LLM calls or filters to catch errors fails in production
“It's both of those things, and I think people are addressing error today by adding more calls to the model of filtering. Out the requests. And I think I don't think that'll work for serious production use cases.”
RAG and prompt engineering are just search techniques, not real AI
“Today when people are running RAG or prompt engineering those are search. That's not AI. It's like keeping the AI frozen and fixed.”
Zhou: Public training data for LLMs is running out
“Yeah, I think even just zooming back out at a technical level, I think actually public data is running out for all, all of what LLMs can take advantage of.”
Zhou: Domain experts, not AI researchers, will drive top models
“However, I believe that, and this is based on my experience training these models, it's actually the domain experts will be driving the best models out there. It won't be people like me who can actually do all the model training, et cetera.”
Zhou: Lamini cuts LLM fine-tuning time from months to milliseconds
“And by efficiency, I mean, you know, it's instead of something that might take weeks or even months that's bringing it down to even like the millisecond level.”
Zhou: Lamini is the only platform running LLMs on AMD GPUs
“We are the only folks who can actually run your language models on top of AMD AMD GPUs.”
Lamini's AMD support unlocks 20,000 GPUs, enough to train GPT-4
“So that unlocks, what that means is that unlocks about 20,000 GPUs readily available today for enterprises to be able to use, and to get a sense of what that means you can train GBD-IV.”
Zhou: Zero-shot LLMs can replace manual human labeling in RLHF workflows
“Which is that why can't it be another LLM or a pipeline of LLMs that can help with that feedback? I think manual labeling is very tedious, especially for our target user, which is a software engineer. And I don't think people should necessarily have to do all …”
Zhou: Outsourcing specialized medical AI data labeling to Scale AI failed
“We tried outsourcing actually to like scale AI, et cetera. None of that worked. It had to basically be me.”
Zhou: LLMs can reach 99% accuracy today with narrow scoping
“I think we can get to that performance today, but it's based on how you scope out the problem. So if it's a very narrow scope, of course you can get that.”
Lamini's hosted service ran exclusively on AMD GPUs for a year
“The Lamini hosted service over the past year has been running on AMD GPUs only. We haven't been running on NVIDIA chips.”
Lamini has achieved software parity on AMD GPUs with CUDA
“We have reached software parity with essentially CUDA.”
Lower LLM latency requires specialized hardware over GPU software platforms
“Unfortunately, I think what people don't realize is that the way to get better latency, like significantly better latency, is actually in the hardware. And that's why we see Grok, G-R-O-Q be able to exceed all these GPU-based inference platforms significantly,…”
NVIDIA A100s and AMD MI300s are readily available, but H100s remain scarce
“A 100 in particular are pretty available. Obviously the AMD chips that we also agnostically work with the MI 300 and MI two fifties, those are available. H 100 still kind of. A little bit harder to get, but you can get started very easily with any of those oth…”
Memory tuning embeds enterprise data to enable near-deterministic factual recall
“To be able to embed facts of your data into the model, so memory tune the model so that it can recall those facts almost deterministically within its probabilistic context.”
Zhou: Best LLMs of the next wave will be enterprise models
“So actually the next frontier for LLMs is in enterprises, and I believe the best LLMs for this next, next wave essentially will be enterprise LLMs.”
Zhou: ML benchmark accuracy fails in production without low API latency
“In machine learning, you know, a lot of AI people are like, yeah, we push the performance of this model, and we define performance as accuracy or, you know, accuracy along these, like, general benchmarks. Maybe it's sixth grade science questions or something. …”
Zhou: Prompting Midjourney images matters more than writing blog posts
“And so we found that just spending some time on mid journey was worth more than spending time on the blog posts. In any single way, like hands down, just like spend a few extra minutes here, prompt engineering or really just like generating an image of your ch…”
Lamini switches across 1,000 fine-tuned models in three milliseconds
“With the technology that we've used with parameter efficient fine tuning and just like efficiency, different efficiency methods, that time to switch across a thousand models is three milliseconds”
Sharon Zhou: Compounding chain errors can be solved by fine-tuning one model
“The ways to reduce it are you take the input of the first thing into the chain, you take the output of the last thing, and you fine tune only one model to do the whole thing, for example.”
Zhou: Generative AI is notoriously hard to evaluate without domain experts
“Generative AI is famously, famously very hard to evaluate. We like have no idea what's good, better, best, unless someone who's an expert in understanding that That use case can tell you that, right?”
Coding AI advanced faster because developers evaluate their own outputs
“And I think that's why, realistically, we've seen many more, like, code agents and applications around code, maybe Texas SQL. We've seen that a lot because the developer Can also evaluate the outputs, right? They are the one in the same person.”
General-purpose LLMs optimize for average internet error, perfect at nothing
“These models, when they're general they're optimizing for what's known as generalization error, or the average error across all examples it sees on the internet. And as a result, it's pretty good at everything, but it's perfect at nothing.”
The AI inference market for startups is a race to the bottom
“My view of inference today, you know, from a market perspective and, you know, startup market perspective is that it's a race to the bottom today for cost. I don't think that's a controversial statement at all. I think people know that it's getting priced lowe…”
Fine-tuning a GPT-3 class model with LoRA provides a 10,000x efficiency boost
“I think for something like a GPD three level model, it's a 10,000 X speed up in efficiency while losing nearly not much at all in accuracy.”
Future AI models will undergo continuous fine-tuning as easily as prompt engineering
“I believe in a future where we're continuously fine tuning these models where it's as easy as prompt engineering and you know, these models continually improve.”
Zhou: Fine-tuning transformed GPT-3 into ChatGPT
“Fine tuning is the technology that got from a research project in 2020 called GPT-III and turned that into ChatGPT, a billion dollar app, right?”
Zhou: Editing search queries on Google is essentially prompt engineering
“I think Google is essentially prompt engineering. When you edit your query to get the results that you want, that's prompt engineering.”
Zhou: ChatGPT guardrails are difficult due to broad scope
“The reason why ChatGPT is hard to put guardrails on is because they're trying to go after every possible use case, right?”
Zhou: Lamini deploys enterprise LLMs on-premise in air-gapped environments
“We're an integrated inference and fine tuning platform for enterprises to be able to run factual LLMs. So essentially LLMs that don't hallucinate on their proprietary data within their secure walls. So we can deploy on premise air gapped, no internet sites. So…”
Zhou: Adding images drove a 10x increase in marketing engagement
“I think one thing that, you know, we were analyzing our data when it came to all our marketing blog posts or like tweets, everything going out, and there was a 10 X increase in anything with an image, right?”