Manifest AI

“We're going to be completely open sourcing all of the pieces that you need to do this metamorphosis yourself”

“After just 10,000 steps of training, which this training one took about two hours, this orange curve, you see that it fully matches the original loss.”

“Power retention is the specific variant that we're about to release. And it basically works by instead of the memory constantly growing, this constantly Growing KVCache. You have a memory that is a fixed size and each new token simply gets compressed into this…”

“In the end, this converges to, I believe, about 35% accuracy on human eval, whereas the star coder baseline was about 30%.”

“Actually, our initial version of power retention was written in Triton, but we realized quickly that it just didn't offer the flexibility to really squeeze the performance that we wanted out of the GPU. So we took a step back and dove into CUDA.”

“We made a tool called Manifesto, where basically you can point it at any repo, including a very large repo, and it will basically fix any in that repo, or do its best to, of course. It's only a three billion parameter model, so, you know, it has its limitation…”

“That's something we plan on doing at the like, thirty billion scale in the coming months.”