embedding models

“Basically it's impossible to use more than ten billion parameters. For embedding models.”

“On the first level bit I would say is that I think it's kind of orthogonal to embedding models and re-rankers to some degree, because even when you have agent chaining, right, you still probably use embedding models as part of the chain, right?”

“And my long term vision here is that some of the software engineering layers on top of the networks will be less and less needed when the networks are more and more clever.”

“However, in the long run, my suspicion is that iterative retrieval will be useful, but it will be a bit less useful as the If the embedding models becomes more and more clever, right? So once the embedding models are more clever, then maybe one run or two runs…”