Assertion Supported AI assessment confidence: 95% certainty 5/5 debate potential 1/5

OpenAI adds third-party model support to its evals product

Christina Huang · DevDay 2025: Apps SDK, Agent Kit, MCP, Codex and why Prompting is More Important than Ever · Oct 7, 2025 · at 16:38

Christina Huang discusses multi-model evaluation features launched at OpenAI DevDay 2025.

0:00 / 0:05exact quote · 6.0s

▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark

“One of the things that we launched today with evals too is ability to use, like, third-party models as well and kind of bring that into one place”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

OpenAI launched support for evaluating third-party models within its evals product.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

Assessment

On October 6, 2025, in conjunction with DevDay and the introduction of AgentKit, OpenAI officially announced expanded evaluation capabilities within its platform tooling, including direct support for third-party model evaluation. Official documentation confirmed native evaluation support for external models (from providers such as Google, Anthropic via AWS Bedrock, Together, and Fireworks via OpenRouter, as well as custom endpoints). Independent press coverage on October 6–7, 2025, corroborated the release.

OpenAI's official announcement 'Introducing AgentKit' (published October 6, 2025) confirmed new evaluation features including 'third-party model support to measure and improve agent performance.'
OpenAI API platform documentation ('Evaluate external models') detailed support for evaluating non-OpenAI third-party models and custom endpoints in OpenAI Evaluations.
DevDay 2025 reporting across Artificial Lawyer, Superprompt, TechCrunch, and VentureBeat on October 6–7, 2025, confirmed the launch of third-party model evaluation capabilities.

Checked against

[1] openai.com [2] superprompt.com [3] openai.com [4] artificiallawyer.com [5] bitzesty.com [6] sim.ai

OpenAI "evals" "third party" OR "external models" OR "evaluating third-party" "openai.com" "Third-party model support" OR "external models" "DevDay" "2025" OpenAI "evals" launch support evaluate third-party models "openai.com" "AgentKit" "evals" "third-party" "openai.com/index" "AgentKit" OR "DevDay 2025" "evals"

assessed as of 2026-09-02

This assessment is an AI reading of a public statement, made against the cited references with the confidence shown. It can be wrong. The quote, the tape and the references are all on this page; if something is off, use report an error below. full disclaimer →

Mentioned

evals product OpenAI third-party models

⚑ Spot an error? Report it

More from Christina Huang

Assertion Supported

OpenAI launches Agent Kit to build, deploy, and optimize agents

“We launched Agent Kit today. Full set of solutions to build, deploy, and optimize agents.”

Christina Huang Oct 7, 2025 ▶ 9:26 DevDay 2025: Apps SDK, Agent Kit, MCP, Codex and why Prompting is More Important than Ever

Insight

Huang: Chat Interfaces Should Be Standardized Drop-in Components Like Stripe Checkout

“Yeah, so it's very similar philosophically, right? So Stripe, you know, can build elements and check out, and not every business needs to rebuild, right, the pieces that are really common, and I think we see the same with chat. We see chat being built over and…”

Christina Huang Oct 7, 2025 ▶ 36:30 DevDay 2025: Apps SDK, Agent Kit, MCP, Codex and why Prompting is More Important than Ever

What-if

Huang: Building Visual Agent Builder in under 2 months required Codex

“For the Visual Agents Builder, we only started that Probably less than two months ago, and that, that wouldn't be possible without Codex.”

Christina Huang Oct 7, 2025 ▶ 41:12 DevDay 2025: Apps SDK, Agent Kit, MCP, Codex and why Prompting is More Important than Ever

Disclosure

OpenAI plans two-way code sync and code execution in Agent Builder

“Eventually, like, that's definitely what we want to do. Maybe you could start off in code. You could bring it in. We'll also probably have, like, ability to, you know, run code in, in the agent builder as well”

Christina Huang Oct 7, 2025 ▶ 13:42 DevDay 2025: Apps SDK, Agent Kit, MCP, Codex and why Prompting is More Important than Ever

Assertion Not publicly verifiable

Huang: OpenAI Customer Support Is Powered by AgentKit

“We use this internally and externally, like our customer support, help.open.au.com already powered on agent kit and then various other like internal use cases as well.”

Christina Huang Oct 7, 2025 ▶ 33:52 DevDay 2025: Apps SDK, Agent Kit, MCP, Codex and why Prompting is More Important than Ever

OpenAI adds third-party model support to its evals product

More from Christina Huang

Turn any episode into a week of clips.