Brown: AI evaluation requires domain experts, not 100,000 generalist labelers
Campbell Brown · How AI Should Handle News, Politics, Medicine, and Mental Health — With Campbell Brown · Aug 26, 2026 · at 24:27
Campbell Brown, CEO of Forum AI, explains her company's methodology for evaluating AI models on complex topics compared to traditional crowdsourced data labeling firms.
0:00 / 0:06exact quote · 6.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Our view is you don't need 100,000 of people. You need the smartest people in a given domain.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Campbell Brown
Brown: AI writes and synthesizes better than many generalist reporters
“AI can write better than a lot of reporters. AI can synthesize information better than a lot of reporters. So if you're not bringing real genuine expertise or original information to the ecosystem, to the conversation, what are you contributing?”
Brown: Enterprise customers will compel AI developers to prioritize accuracy over engagement
“What's interesting with AI is if you look at what the companies are doing today, which is going after enterprise, that is where the business is, that's where the money is, and I'm a big company spending millions and millions of dollars with Anthropic or OpenAI…”
Brown: Leading AI labs lack independent verification of model safety
“The other piece is the accountability, is there's no independent verification of how the models perform. What we get when it comes to, you know, how, how does OpenAI, or how does Anthropic do on questions around bias, what we get from them is a blog post that …”
Brown: Enterprises have a problem if AI vendors grade their own outputs
“If you are using AI for something really important, ask yourself, who is checking the outputs? And the information that you're getting. And if it's the company that sold you the AI, you have a problem. So you, whether you're building your own evals internally …”
Brown: OpenAI shifted focus to enterprise after consumer lagged
“The consumer version of this has not been driving the way that enterprise has. And, you know, OpenAI started down that path and then realized, you know, from a business perspective, they needed to focus on enterprise and shifted. And now that's what they're do…”