The Ledger, every show

“Been able with memory tuning, which is what I've been working on to remove those hallucinations, to remove that and actually get these models from, you know, not necessarily being general for everything. And instead of being pretty good at everything, but perf…”

“Today, actually, I'm seeing the GPU shortage go away at the level, at the company level, meaning companies are able to procure enough compute enough is a strong word, but they're able to procure compute at some level to work with, to fine tune and run heavy in…”

“So that's something else we offer through our inference service to actually make it a hundred percent by re-engineering the decoder of any LLM.”

“I even think there's a future where these models can be a hundred billion parameters, but at, you know, have that intelligence of a hundred billion parameters, but then have the speed, latency, and cost of something that's still one billion or seven billion pa…”

“And I do think that's the future so we can get something that is incredibly smart, incredibly huge, but with the latency cost and speed of something, something tiny. So no more big model versus small model paradigm. It's potentially one in the same.”

“Yeah, I think even just zooming back out at a technical level, I think actually public data is running out for all, all of what LLMs can take advantage of.”

“However, I believe that, and this is based on my experience training these models, it's actually the domain experts will be driving the best models out there. It won't be people like me who can actually do all the model training, et cetera.”

“And by efficiency, I mean, you know, it's instead of something that might take weeks or even months that's bringing it down to even like the millisecond level.”

“We are the only folks who can actually run your language models on top of AMD AMD GPUs.”

“So that unlocks, what that means is that unlocks about 20,000 GPUs readily available today for enterprises to be able to use, and to get a sense of what that means you can train GBD-IV.”

“We tried outsourcing actually to like scale AI, et cetera. None of that worked. It had to basically be me.”

“I think we can get to that performance today, but it's based on how you scope out the problem. So if it's a very narrow scope, of course you can get that.”

“We have reached software parity with essentially CUDA.”

“A 100 in particular are pretty available. Obviously the AMD chips that we also agnostically work with the MI 300 and MI two fifties, those are available. H 100 still kind of. A little bit harder to get, but you can get started very easily with any of those oth…”

“So actually the next frontier for LLMs is in enterprises, and I believe the best LLMs for this next, next wave essentially will be enterprise LLMs.”

“With the technology that we've used with parameter efficient fine tuning and just like efficiency, different efficiency methods, that time to switch across a thousand models is three milliseconds”

“I think for something like a GPD three level model, it's a 10,000 X speed up in efficiency while losing nearly not much at all in accuracy.”

“I believe in a future where we're continuously fine tuning these models where it's as easy as prompt engineering and you know, these models continually improve.”

“Fine tuning is the technology that got from a research project in 2020 called GPT-III and turned that into ChatGPT, a billion dollar app, right?”

“I think one thing that, you know, we were analyzing our data when it came to all our marketing blog posts or like tweets, everything going out, and there was a 10 X increase in anything with an image, right?”