DeepSeek

“China was kind of not in this fight, like, 12 months ago, and now is very much in it. Like, their models like Alibaba's Quen and then this spin out from a quantitative hedge fund DeepSeek, which publishes code models and others, and they've been actually a ver…”

“We have kind of an existence proof now that it's actually not that hard to do this and so you don't need to invest all that much in, in data, and you can use synthetic data and get a pretty good model out of that”

“So I would guess that they spent at least a hundred X The amount of that, that single training run, right?”

“We have been working on function for calling for a long time, and we are the first one to enable function calling for deep seek models.”

“DeepSeq for example, just within one month of releasing their new models, There are, despite DeepSeq model, extremely hard to tune and optimize, extremely hard. There are 500, more than 500 variants published on Hugging Face, optimizing for local device, optim…”

“We actually hosted the full version of DeepSeek, not their small model.”

“If you were to compare them to XAI, I would say they are definitely one or two steps behind in terms of their ability to come from nothing and train a world-class foundation model.”

“For me, there was no technology DeepSeek moment.”

“We had a bunch of cloud credits and I, they were running out and we're behind and I just generated like as many completions as possible. So it was like a few billion completions from deep seek over the weekend.”

“If you look at the deep seek model, for instance, this is one of the best open source models we have out there today. It was trained at the end of 2024. On last generation, kind of nerfed GPUs, H 800 instead of H 100, the 800 is nerfed by all sorts of ways fro…”

“And you can actually, in fact, Take a GPT one or two model and with a few, I mean, few lines of code almost, you can transform it into the latest let's say deep seek version, 3.2 architecture. It's not like a big leap. It's still the same as scaffold.”

“Now when you look at reinforcement learning from models like Kimi or from DeepSeq models, it seems that they are closer to one million data points.”