Artificial Analysis, every mention
18 scenes (2026), the whole family · ← back to Artificial Analysis
tap a year for its mentions
every year 2026 anyone Shawn Wang 31George Cameron 16Micah Hill-Smith 14Andrew Feldman 4Stefano Ermon 2Lin Qiao 2Pranav Reddy 1Philip Kiely 1Barak Lenz 1
Verbatim, from the transcripts: the passages where Artificial Analysis comes up
Next 100x in AI: Inference, Networking, & Self-Optimizing Models — Philip Kiely & Ali Taha, Baseten
- ▶ 38:14 Philip Kiely Um, so the spreads that you see, if you like go on artificial analysis or you go on open router and you look at, you know, the, the worst provider to the best provider oftentimes can, can hit that kind of range.
The AI Frontier: from open weights to open research — Eiso Kant, Poolside AI
- ▶ 1:07:39 unnamed speaker Um, and so, I mean, the, the other way is just to give it to artificial analysis and let them run it like third parties. 2 times in the scene
The Agent Cloud: Databricks’ Bet on the Future of AI — Matei Zaharia and Reynold Xin
- ▶ 28:26 Shawn Wang Uh, I do think like there are a bunch of like artificial analysis is obviously, uh, doing super well, uh, with, with their stuff.
Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
- ▶ 0:09 Micah Hill-Smith Um, cause the like first time artificial analysis got mentioned on a podcast was you and Alessio on Land and Space. 2 times in the scene
- ▶ 4:16 Shawn Wang Um, what was the impetus for starting artificial analysis in the first place? 4 times in the scene
- ▶ 7:49 Shawn Wang The way that if I were to start artificial analysis at the same time you guys started, I would have used the Luther AI's, um, eval framework, uh, harness.
- ▶ 11:58 Micah Hill-Smith So like one of the things that we do is run an enormous number of all of our evals when we're developing new ones and doing upgrades to our intelligence index to bring in new things. 3 times in the scene
- ▶ 17:20 George Cameron Uh, and they've been great mentors to us as we've built artificial analysis. 4 times in the scene
- ▶ 19:13 Micah Hill-Smith So first, just like background, like we're talking about the artificial analysis intelligence index, which is our synthesis metric that we pulled together currently from 10 different eval data sets to give what? 3 times in the scene
- ▶ 22:23 George Cameron Currently, the Intelligence Index and evals focus a lot on kind of raw intelligence, but we kind of want to diversify how we think about intelligence, and we can, we can talk about it, but kind of new evals that we've kind of built and… 6 times in the scene
- ▶ 25:01 George Cameron But, um, something that I actually don't think every artificial analysis user knows is that you can customize our charts and, and choose what, uh, what models are highlighted.
- ▶ 26:55 Shawn Wang so this is the, the rough, uh, pitch of AAQI, uh, or is it AAQI or AAII? 2 times in the scene
- ▶ 33:14 George Cameron One eval that we added to artificial analysis is, is, is critical point, and it's really hard, uh, 2 times in the scene
- ▶ 51:10 George Cameron Yeah, we've looked at it from a evals perspective, and we love Terminal Bench and, and host benchmarks of Terminal Bench on artificial analysis.
- ▶ 52:11 Micah Hill-Smith Like long context reasoning data said last year, we did open source. 3 times in the scene
- ▶ 58:35 Micah Hill-Smith A couple of like the big trends that we track at artificial analysis over time and that like, we're always showing charts of on the trends page and these reports and stuff. 3 times in the scene
- ▶ 1:14:16 George Cameron And I'm not sure that that's, that's the case, but I think for artificial analysis, we'll keep benchmarking, uh, raw intelligence, but we also want to think about it and explore models more deeply across other axes as well. 2 times in the scene
- ▶ 1:14:52 Shawn Wang So what is going to be included in, let's say like a V three of the intelligence index, because obviously you're going to saturate. 6 times in the scene