Reflection AI

“So we test them. We use them. We try to we try to learn from them. We also partner with US companies that are trying to create open source models. There's actually a company that's based in in, in Brooklyn and San Francisco that is that, that, that we work wit…”

“It was reported yesterday by the misinformation that reflection, reflection the misinformation reported that reflection one-year-old startup we've had the CEO on before he's in talks to raise one billion to develop open source models to compete with deep sea m…”

“We realized that probably the form factor of an organizational super intelligence, like the thing that's really gonna help organizations get a lot of stuff done, is probably gonna be something like an oracle. Like an oracle that understands the entire organiza…”

“It'll grab it, and then that's all you have, and most likely, for any meaningful query it will not have given you the information that you need to actually go do the task. So rag agents are actually, are pretty weak.”

“That's basically where Agentex search is today, where you're going and you're kind of exploring, and you have this tiny flashlight, and you have to then remember everything in your head, and obviously that's a first form of comprehension, but it's also a prett…”

“I'm sure that other companies will converge on it as well.”

“It's really easy to switch around between various different coding providers today because they don't really integrate deeply, and so, you know, you can try cursor today, you can try cloud code tomorrow, you can switch, you know, to windsurf, and there's reall…”

“There is in some sense, we are probably at a L four kind of junior, junior engineer level of autonomy, which is pretty incredible.”

“And that the combination of this you know, L-Nine with Amnesia and the L-Nine's context core will together, you know, that will become the principal level engineer, the AI engineer. And so I actually think that that's not too far away. That's I would say in, y…”

“We win over candidates over OpenAI and Anthropic Meta, DeepMind regularly.”

“Later this year we'll be shipping things that I don't think anyone ever thought a startup could do. Like, I think that we're going to be shipping some things on the research side that I think everyone thinks you need to be a giant lab with a 100,000 GPUs to do…”

“To some extent super intelligence in that sense has already been achieved. So right, AlphaGo was a super intelligent system, and there were other systems during that time that were built that were super intelligent in narrow domains.”

“As long as you pick a category that I would say is kind of big enough to be ASI complete I think, and this is kind of our approach at Reflection, is it makes a lot more sense to be focused and co-design those two things together, the product of the research.”

“If you're looking at a file, and there's like a specific thing in that file, and you're just trying to get a quick answer to it, you don't really need the hammer of like a deep research like experience. You don't need to wait, you know, like tens of seconds or…”

“Where if you want to change the agents, the team wide memory, then it probably is going to look something like a pull request where the person who really understands that system Approves or, you know, edits it or something like this. I don't think it's going t…”

“We're in this brief period in history right now where the RL flops are still manageable. Like you can really have a best in class product if you're focused. And yes, you'll need to put, you know, you still need a decent amount of GPUs, but from a flops perspec…”

“Our thought was that this was the time where you can actually start a you know, a generational frontier lab that does not need to be coupled to a, you know, to a big cloud provider because if you do it right, you'll actually be able to generate you know, suffi…”

“I don't think it's guaranteed that a big lab can, you know, buy their way to the end user because the fundamental problems of your, you know, research team being far away from your product team will still be true. And the company having, you know, a hundred di…”

“And then from the startup side, I think it actually puts companies that Are in these kind of critical path categories like search and coding in a pretty existential place if they can't build their own frontier models. Not all frontier labs will be able to vert…”

“The reason code is special is if you believe that the way a language model will interact with almost any piece of software is through function calls and therefore code, then if you build very capable reasoners coding reasoners that, you know, are sort of purpo…”

“I think coding is this era as well. This one I think will take longer than people thought as well, because again, enterprise is organizational problems. There's much different than The benchmarks that we have today, but I think it will be one of the faster one…”

“Our core belief is that if you solve this problem, you solve the autonomous coding problem and build a super intelligent coding agent, that that thing will lead to super intelligence more broadly.”

“Our belief as a company is that the correct wedge in, the correct starting point to this entire problem is decoding agent, because it's already, you know, software engineering is already what I would call kind of ergonomic for a language model.”

“It can potentially work otherwise, but practically it only works when the agent has interacted with a reward, right? It's received a positive reward for what it's done. Maybe one out of 10 times, one out of 50 times, but if it's getting zero reward, then you d…”

“I think I think there are going to be a lot of move 37”

“So it's not just having long context, it's whether your model truly understands what's inside the context, and needle in the haystack tests are pretty crude and not very effective way of testing this kind of capability.”

“I bet would be on long context understanding and improving the attention mechanism over the long context.”

“Autonomous coding benchmarks, let's say, like Sweetbench, are useful. I'm not going to discount them. They are useful. But let's say, you know, 90% on Sweetbench could still mean something that just falls over flat within a customer setting.”