Pre-December 2024 OpenAI models did not exhibit seahorse emoji self-correction loops
Pratyush Maini · ⚡️ Reverse Engineering OpenAI's Training Data — Pratyush Maini, Datology · Feb 10, 2026 · at 9:04
Pratyush Maini of Datology explains when recursive self-reflective behavior began appearing across OpenAI's model release timeline.
0:00 / 0:25exact quote · 26.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And so I like ran the OpenAI API across like models released from 23 to 25, and you would see like all the models until twenty-twenty-four December had very Terce and short responses to the question. Is there a seahorse emoji? They would either say that there exists. Sometimes they would say they do not exist, but they would never go into this self correction loop where they like say one answer in the beginning and then a different answer at the end.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Pratyush Maini
Modern LLMs verbatim regurgitate JEE exam questions from two-word prompts
“We consistently saw how many of these, like, models today are being, like, massively, like, kind of fine-tuned on problems from... Like, oversight? Massively worked with. Like, even, like, imagine if I ask you the light bulb, what comes next in your mind? It w…”
Labs train LLMs on benchmark questions for multiple epochs late in training
“It's very clear how the last stage of training for many of these models does have a massive amount of example or examine. Benchmaxing. Because the model will not, like, behaviorally complete exam questions with options if they have not really seen it at the en…”
Self-reflection training data is now core to all frontier foundation models
“What this suggests about the GPT training data is that the self-reflection data has now actually become pretty much core to the training of all frontier models, because we're seeing that happen in non-instruct models across the board.”
Core AI capabilities must be built during pre-training, not just fine-tuned
“If there is a core capability that you actually care about, that capability should be part of the foundation and not a fine-tuned artifact.”