AI Inferencing
topic on 2 shows · 2 statements across 2 episodes
2 statements about AI Inferencing, every show
Jain: Open source models will dominate AI inferencing within two years
“Well, I think open source models are going to dominate AI inferencing. You know, you'll see in the next two years, we'll shift to where's almost no open source to where's going to be almost all open source.”
McAvoy: Grid response requires site-level inferencing for sub-50ms latency
“The goal right now, I think for a lot of people, it's like sub-fifty millisecond response, and so you want to reduce latency everywhere, so you start doing your inferencing and decision making, like, locally, very locally, on the ground, like, at the site leve…”