Opinion
Lenz: Model providers should not dictate enterprise AI policies
“Right now, if you're using a model, you're taking in their own policy. Even if I want to use GPT-OSS, I've taken in a lot of different policies about what to abstain from, what's considered dangerous and not dangerous, how I should behave, etc. And I don't thi…”
Assertion Not checkable as stated
Lenz: Local smartphone AI requires hybrid models due to KV cache limits
“So if you wanted to do something local on your phone to search your images, as an example, you can't do that without a hybrid architecture or without doing drastically changes because the model plus KVCache won't fit.”
Opinion
Lenz: Most enterprises avoid reasoning models due to high latency
“Most enterprises don't really want to use reasoning models. The latencies is too high”
Prediction Not checkable as stated
Lenz: Hybrid Transformer models are here to stay for long context
“If I had to guess hybrid models are here to stay just because the efficiency without sacrificing the performance is, is too much to give up. You know, the attention is so expensive. The quadratic cost and the linear memory cost is so much that I think for long…”
Insight
Lenz: Middle attention placement at 1:8 ratio optimizes hybrid models
“Putting it the first or the last performed worse than in the middle. And one to eight was good enough. You know, you might get very slight improvements with one to six, but it was marginal, maybe within the standard deviation.”
Prediction Not checkable as stated
Lenz: Full attention models will decline as sequence lengths rise
“I can definitely see sequence length rising, and I can't see full attention models being as prominent as they are today. So, so, so at least they'll have less full attention layers and I hope they'll have more innovations like Mumbai.”
Assertion Not checkable as stated
Lenz: No open-source infrastructure can train very large models
“And we're using our own infrastructure to train our models. There still isn't an open source infrastructure that I could say, use this to train your very, very large model.”
Insight
Lenz: Engineers seeking online answers are not at the frontier
“And I'm looking for people that try to solve problems on their own. Because if you think you're gonna found the answers online, I think you're not in the frontier.”
Insight
AI21 CTO: AI systems must be model-agnostic and action-oriented
“AI systems need to be model agnostic. They shouldn't care about which model that they use, and they should look at what I call actions, which is a combination of a model with a prompt and maybe a set of tools that it can use and say, what can an action do for …”
Insight
Lenz: RL training wastes compute on saturated or impossible examples
“Once you've trained a few hundred steps of let's say GOP, Most of your training is just wasted on example that are either too hard for you and you didn't get any success on them or too easy and everything was a success.”
Assertion Contradicted
Lenz: AI21's Jamba is the first hybrid model architecture
“Since then, we've released several models, recent model lines in called Jamba, which I think the fascinating part about it is, is the first hybrid model. It's not just attention.”
Assertion Partly supported
AI21 CTO: Jamba 3B uses 1:12 attention ratio to cut memory footprint
“So this is a three B dense with only two attentional layers. It's one to 12 and not one to eight, because we wanted to maximize the efficiency. It has very few attention heads, so everything is geared To have, you know, long context with very little memory.”
Insight
Lenz: 3B models cannot be generalized and must be task-tailored
“Because again, three B models, they need to be tailored for tasks. It's not like you could squeeze whatever you wanted into them.”
Insight
Lenz: Training foundation models mirrors developing algorithmic trading strategies
“So, so it's very similar in terms of how you interpret results. You want to treat everything as a black box. You want to establish your bounds, you know, what are you, and I've had tons of experience in algotrading, both from making money and not making money …”
Assertion Not checkable as stated
Lenz: GPT-4o is an AI orchestration system, not a raw model
“GPT-IV-O is already an AI system. It's not calling a model directly. It can do certain, you know, it can do tool calls. It can orchestrate this entire thing.”
Disclosure
Lenz: AI21 will release a 3B dense Jamba model for edge devices
“And we're actually releasing new versions of our Jamba models soon, including a three B dense model. It's aimed for long context on edge devices.”
