StarZero All shows Try the podcast clipper →

George Cameron

Co-Founder & CPO, Artificial Analysis · 1 appearance on the record.

computed by AI from the episodes · how this works → · full disclaimer →

founderexecutive@georgecameron ↗artificialanalysis.ai ↗

George Cameron is the co-founder and Chief Product Officer of Artificial Analysis, an independent AI benchmarking house tracking model capability, latency, and economics. He was previously a senior strategy consultant at Altman Solon specializing in data centers and technology strategy.

11statements → 7claims → 4claims resolved → 75%fully supported → 3.73/5average certainty → 2.18/5average debate potential →

3 supported 0 partly supported 1 contradicted 1 not yet assessed 2 not checkable as stated how the 7 claims stand · each chip opens the sources

1 prediction · 6 assertions · 1 opinion · 2 insights · 1 disclosure · every statement was checked. The prediction and assertions are the 7 claims: statements the public record can support or contradict. 4 are resolved, 1 is not yet assessed, and 2 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.

The record, in short

What the tape says about how George argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.

Their most notable supported claim

Assertion Supported
Cameron: General model intelligence does not correlate with hallucination rates
“One interesting aspect is that we've found that there's not really a, not a strong correlation between intelligence and hallucination rate. That's to say that the smarter the models are in a generalist sense isn't correlated with their ability to, when they do…”
George Cameron Jan 9, 2026 ▶ 31:28 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith

Their most notable contradicted claim

Assertion Contradicted
Frontier models are cheaper for agentic tasks because they require fewer turns
“Interestingly, in Tau Tau Two Bench Telecom, it's cheaper to run, you know, on a per token basis, more expensive models, like a GBD five, compared to some smaller open source models, because the some of the GBD five, for instance got to the answer faster. And …”
George Cameron Jan 9, 2026 ▶ 1:09:14 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith

How they sound: not measured why? →

We measure speaking style by listening to the audio itself, and a fair number needs at least 2,000 words from one person on tape we have measured. There is too little of George Cameron on measured tape to publish a rate. This says nothing about how they speak.

Everything George Cameron said on Latent Space that made the record, most notable first. Filter by type, assessment or year in the ledger →

Assertion Supported
Cameron: General model intelligence does not correlate with hallucination rates
“One interesting aspect is that we've found that there's not really a, not a strong correlation between intelligence and hallucination rate. That's to say that the smarter the models are in a generalist sense isn't correlated with their ability to, when they do…”
George Cameron Jan 9, 2026 ▶ 31:28 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Assertion Not publicly verifiable
Models perform better in custom agent harnesses than native web chatbots
“And what's really interesting is that if you compare, for instance, Claude, 4.5 Opus using the Claude web chatbot, it performs worse than the model in our Agentic harness. And so in every case, the model performs better in our agentic harness than its web chat…”
George Cameron Jan 9, 2026 ▶ 45:40 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Insight
Cameron: Models perform better with minimal tools than rigid frameworks
“I think where we're getting to is that these models have gotten smart enough, they've gotten better, better tools that they can perform better when just given a minimalist set of tools and let them run, let the model Control the agentic workflow rather than us…”
George Cameron Jan 9, 2026 ▶ 51:27 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Assertion Supported
Cameron: Model performance correlates with total parameters, not active parameters
“We, in our benchmark, see a lot of performance correlated more with total parameters than active, and not that correlated with how sparse like the models are. Our accuracy benchmark is part of a omniscience. It's very correlated with total. It's not correlated…”
George Cameron Jan 9, 2026 ▶ 1:05:08 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Assertion Contradicted
Frontier models are cheaper for agentic tasks because they require fewer turns
“Interestingly, in Tau Tau Two Bench Telecom, it's cheaper to run, you know, on a per token basis, more expensive models, like a GBD five, compared to some smaller open source models, because the some of the GBD five, for instance got to the answer faster. And …”
George Cameron Jan 9, 2026 ▶ 1:09:14 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Opinion
George Cameron: Mixtral 8x7B transformed the landscape for serverless inference providers
“We had Mixtrel A times seven B and it was a key. Like a open source model that really changed the landscape and opened up people's eyes to other serverless inference providers and thinking about speed, thinking about cost.”
George Cameron Jan 9, 2026 ▶ 6:32 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Insight
Cameron: Inference economics incentivize larger, sparser AI models over dense architectures
“It's, I think, less about total parameters in many cases when thinking about inference costs and more around number of active parameters, and so there's a bit of an incentive towards larger, sparser models.”
George Cameron Jan 9, 2026 ▶ 38:16 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Assertion Not checkable as stated
Gemini 3 Pro performs poorly on GDPval-AA benchmark evaluator tasks
“One data point there is that even as the, as an evaluator, Gemini three pro interestingly doesn't do actually that well in GDP val AA.”
George Cameron Jan 9, 2026 ▶ 41:41 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Prediction Not checkable as stated
Cameron: Turn count will become a major AI benchmarking metric
“I think number of turns is, is going to be a metric that we're going to be talking about a lot more. And I think it'll be something that people want to really start to think about a lot more.”
George Cameron Jan 9, 2026 ▶ 1:09:46 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Disclosure
Artificial Analysis open-sources minimalist agent harness Stirrup on GitHub
“We released that on, on GitHub yesterday. It's called Stirrup, so if people want to check it out, and it's a great you know, base for, you know, generalist building a generalist agent.”
George Cameron Jan 9, 2026 ▶ 50:12 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith
Assertion Supported
AI2's OLMo 3 32B leads Artificial Analysis's 18-point Openness Index
“It's out of 18 currently. And so we've got an openness index page, but essentially these are points. You get points for being more open across these different categories and the maximum you can achieve is 18. So AI two with their extremely open OMO three, 32 B…”
George Cameron Jan 9, 2026 ▶ 53:35 Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hill-Smith

Appearances (1)

EpisodeDateSpeaking time
Artificial Analysis: The Independent LLM Analysis House — with George Cameron and Micah Hi Jan 9, 2026 14m
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more