why aren't all 8 resolved? a statement only gets an assessment when the public
record can support or contradict it. opinions and what-ifs never can, and 0 checkable
ones are still open, waiting for their date. predictions held up or didn't;
assertions are supported or contradicted. on every card:
▮▮▮▮▮ certainty ·
▮▮▮▮▮ debate potential. speakers are clickable
Insight
AI kernel models game automated evaluation metrics with subtly incorrect code
“It would be great, but it's not a silver bullet because kernels are also hard to validate. It's hard to have like an eval in kernels. Every time that someone releases like, oh, we created a new eval that measures kernels and we trained a model that generates k…”
Assertion Supported
AMD MI300X outperforms Nvidia H100 on FlashAttention-2 and memory-bound workloads
“We found that it's great for flash attention to specifically, we were able to be H-one hundred. We also found that like the less time you spend in like dense compute, like the less time you spend in tensor cores specifically, or less time you spend in lower bi…”
Insight
String theorists ramp faster in AI engineering than complacent CUDA developers
“I don't really care if someone knows CUDA kernel writing. If someone does string theory and is really good at understanding complex problems, they will be productive faster than someone who knows CUDA and doesn't really care about trying to get better at it.”
Disclosure
Zyphra moves its entire model training cluster to AMD hardware
“We recently moved all of our training cluster over to AMD. So we're really going all in on AMD ecosystem.”
Disclosure
Zyphra strictly bans engineering candidates from using AI tools during interviews
“No AI in the interview, first off. And you gotta watch people's eyes now on what monitors they're looking at behind the screen, which is, that's changed in the last two or three years, unfortunately. No AI allowed.”
Insight
Writing custom GPU kernels is a last resort in model optimization
“Well, kernel is a last resort. So first I go, oh, another kernel. And I try and find some way to go around it.”
Insight
AI coding assistants degrade quickly on low-level CUDA and PTX code
“Getting Below Triton or level or anything else down to like CUDA or below there's orders of magnitude less public good kernels at that level. And I think that really shows and models capabilities. So when I try and get a model to do something in CUDA or PTX or…”
Assertion Supported
Zyphra: Zamba 2 7B beats Llama 3 8B using hybrid Mamba-Transformer architecture
“We released a Zamba two, which was a hybrid between transformers and a Mamba two blocks. And we were able to be like a Lama three eight B for example, with a seven B model.”