The Ledger, every show
Every statement that passed quotation and attribution checks, across all 44 shows. Pick shows below, then mix any filter with any other.
shows 
every show 44 of 44
Tay: Most specialized tools will be subsumed directly into model parameters
“Then the most I can see in the future is there'll be a model then that, that is, there's something that really cannot be subsumed by a model. Then you just use a tool or something, right? But my prediction is that I think most things can be subsumed by the mod…”
Yi Tay: The Architecture That Achieves AGI Will Still Be a Transformer
“It will be a transformer, I think. Like people, it depends on what you call it, but I think unless the paradigm shifts completely, which is, I mean, as a scientist, you cannot like completely say no to like that, this would never happen. But my feeling is that…”
Tay: Zero-shot benchmark scores at 1B model scale are random chance
“Every time some people propose like this, they run like some zero-shot score on like some LM event harness or something like that, and you know like at one B scale, all the numbers are random, basically. Like all your bull kill, they're all like random chance …”
Gemini's IMO Model Checkpoint Required Only One Week of Training
“The training process of this IMO model itself was, like, maybe a week or so.”
Yi Tay: AI Model Laziness and Edge Flaws Will Disappear via General Scaling
“I don't think there's anything that to be done to specifically like focus fire. These things is more like general capability improvements. The models just get better over time and then these things will just like go away.”
Tay: Google and OpenAI built general models three years before academia
“Places like Google and Meta, OpenAI, we will be working on things, like, Three years ahead of everybody else, and then suddenly, like, then Academia would be, like, still working on, like, these task-specific things.”
Tay: Multimodal AI architectures will eventually move completely to early fusion
“As early fusion models get more traction, I think the themes will start to get more and more, like, it's a bit like how all the tasks like unify, like from Like, two zero one nine to, like, now it's like all the tasks are unifying, now it's like all the modali…”
Tay: Vision models will unify screen intelligence and natural imagery without bifurcating
“I think at the end of the day, like, the models would become, like, I don't see that there will be, like, screen agents and, like, natural images. Humans, like, you can read what's on a screen, you can go out and appreciate the scenery, right? You're not, like…”
Yi Tay: AI Labs Prioritize Data Efficiency Because the World Lacks Tokens
“I think in general, the, like learning more, like extracting more from varied data points is definitely valuable, but I think that's what related to the fact that we're like running out of Tokens in the world.”
Yi Tay: UL2 is a 20B encoder-decoder model using T5 pre-training data
“So I think UL-II is an encoder, decoder, 20 B model. I think when we got it approved, it was, like kind of, you know, it was released as, like, kind of, like, the big brother of T-Five, you know, kind of like, okay, we updated T-Five with, like, a new objectiv…”
Yi Tay: Jason Wei originated the Emergent Abilities paper thesis
“That was mostly Jason's thesis, and I have to really say that, like Jason has really good ideas, and I was more of, like, a support role for that paper, yeah.”