transformer architecture

“So I would say, and maybe I'm biased because one of my best friends is, is on the original attention paper, but that was the real breakthrough. It's just like figuring out that you have this attention mechanism that actually allows you to yeah, to do a much be…”

“No, they jumped all over it. So it went to production inside of search, inside of translate, like the existing product suite. And so they, to say that they didn't adopt the transformer architecture would not be correct.”

“I would say right now, yes, because it's still the state of the art. So there is nothing really better in terms of state of the art performance, getting better quality results.”

“There was also benefit of doing that simply because the rest of the, that the machine learning field, which was working on, on, on language, they were using this, like architecture. So they were building infra for it, making it faster. And, you know, like the,…”

“I think if we hadn't invented the transformer, we would have gotten there with whatever LSTM you know, state space model, whatever, anything else people were developing, we would have gotten there.”