Jan 2, 2024 · 1h 9m · latent-space
The AI-First Graphics Editor - with Suhail Doshi of Playground AI
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
Suhail Doshi, founder of Mixpanel and Playground AI, discusses the engineering and interface principles behind generative image models, detailing the training of Playground v2, the need for canvas-based graphics editors, and the challenges of scaling AI infrastructure.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
The hosts as informed peer 5.2 Guest teaching 3.7 Guest disagreement 2.3 The hosts pushing back 2.0
speaking balance: gold is the hosts, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 26:45 Rejecting pipeline ensembling in favor of unified architectures
Suhail forcefully rejects the host's assertion that AI art pipelines are the standard solution, labeling pipeline models as computationally expensive hacks that fail to generalize.
Hardest push from the hosts ▶ 46:35 Challenging NSFW dataset filtering logicThe host pushes back against Suhail's dataset filtering choices by raising research showing NSFW images significantly improve a model's ability to render human anatomy accurately.
Biggest teaching moment ▶ 4:50 Contrasting single-threaded CPU limits with parallel AI computationSuhail educates the hosts on why browser virtualization failed due to single-threaded JavaScript CPU plateaus, whereas deep learning models unlock true remote compute scaling via massive parallel matrix arithmetic.
The host holds their own ▶ 17:35 Dissecting open-source research translation speed and technical toolingAlessio demonstrates strong domain mastery by drilling into Playground V2's ground-up training methodology, benchmark metrics against SDXL, and technical trade-offs in community fine-tunes.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | The hosts as informed peer | Guest teaching | Guest disagreement | The hosts pushing back | Why |
|---|---|---|---|---|---|---|
| Mixpanel Machine Learning Reflections and Early Predictive Experiments | 4 | 2 | 1 | 1 | The hosts inquire about Suhail's early ML initiatives at Mixpanel around 2015-2016. Suhail openly shares historical context about their logistic regression experiments and learning ML via fast.ai, while the co-host connects this to user feedback loops. | |
| The Mighty Experiment and Shifting Compute to AI | 5 | 3 | 2 | 2 | Alessio asks about the architectural parallel between remote browser streaming at Mighty and modern AI cloud inference. Suhail breaks down why single-threaded JavaScript CPU bottlenecks doomed Mighty, whereas AI models are fundamentally designed for massive parallel computation. | |
| Transitioning into Generative AI and Founding Playground AI | 5 | 3 | 2 | 2 | Suhail walks through the ideation maze after Mighty, including direct discussions with Sam Altman and Aditya Ramesh about whether OpenAI would develop a specialized graphics UI. The hosts prompt him on business models and customer profiles, and Suhail explains his strategy starting with hobbyists. | |
| Developing Playground V2 and Advancing Open Source Research | 6 | 4 | 2 | 2 | Alessio brings technical context regarding Playground V2's training from scratch and its comparison to SDXL. Suhail corrects the host's assumption about XL Turbo's architecture and explains the rationale for releasing lower-resolution pre-trained weights for compute-constrained researchers. | |
| Model Quality Optimization and Unified Architectures Versus Pipelines | 6 | 5 | 4 | 3 | The host points out that current industry practice relies on multi-stage pipeline models to fix facial artifacts like eyes. Suhail pushes back firmly, arguing that pipeline models are computationally inefficient, ungeneralizable hacks, and advocates for end-to-end unified architectures. | |
| Benchmarking Aesthetic Performance and Measuring Real World Quality | 6 | 5 | 3 | 3 | The hosts and Suhail discuss aesthetic benchmarks like MJHQ and critique standard automated metrics like FID. Suhail explains the necessity of benchmarking against industry leader Midjourney and collecting live crowd-sourced human preference data from active users. | |
| Text Synthesis in Images and Navigating Content Safety | 5 | 4 | 3 | 2 | The host questions the absence of text synthesis in Playground V2 and probes whether training on NSFW data improves body anatomy synthesis. Suhail reframes the issue from training efficacy to safety filtering and the societal risks of election deepfakes and consistent non-consensual generations. | |
| Reimagining the Graphics Editor and Canvas Interface Design | 6 | 4 | 2 | 2 | Alessio highlights Playground's canvas UX differentiators, including seed selection and bounding box outpainting. Suhail explains the rationale for preview rendering via LCMs and discusses why LoRAs remain a temporary technical bridge until promptable moodboards exist. | |
| GPU Infrastructure Management and Scaling Distributed Training Systems | 5 | 4 | 3 | 2 | The host asks about running bare-metal GPU infrastructure and suggests off-the-shelf management tools like Mosaic. Suhail details the fragility of large distributed training clusters where node dropouts cause run crashes and explains why they rely on custom Slurm and PyTorch stacks. | |
| Future AI Modalities and Advice for Self-Taught Founders | 4 | 3 | 1 | 1 | The interview concludes with speculative discussions on future physics foundation models versus consciousness modeling, followed by practical advice for self-taught AI builders to learn via Andrej Karpathy's video lectures and hands-on coding rather than textbooks. |
Statements from this episode (20)
Doshi: Mighty was abandoned because single-threaded JavaScript capped its potential
“One of the problems with Mighty was that JavaScript is single threaded in the browser. And what we learned, you know, the reason, reason why we kind of abandoned Mighty was because I didn't believe we could make a new kind of computer. We could have made some …”
Doshi predicts AI will split between local models and data centers
“Yeah, I think there still will be, you know, local models and then there'll be these very large models that have to be run in data centers.”
Doshi: High-utility generative graphics tools will ultimately be B2B products
“I think ultimately if you believe that you could make it very high utility, the, probably the next customers will end up being B to B. It'll probably not be like consumer. Like there are, there will certainly be a variation of this idea that's in consumer. If …”
Doshi: Image Generative AI Is Stuck in a 'GPT-2 Moment'
“So I think that we continue to feel like graphics and these foundation models for anything really related to pixels, but also definitely images continues to be very under invested. It feels a little like graphics is in like this GPT two moment, right? Like eve…”
Playground AI released 256px and 512px pre-trained weights for Playground v2
“And we also gave out pre-trained weights, which is very rare. Usually you just get the aligned model and then you have to like see if you can do anything with it. We actually gave out there's like a two 56 pixel pre-trained stage and a five 12. And we did that…”
Doshi: Image generation will ultimately converge on unified models
“I think we will make an unified model. I think it will, I think we'll certainly in the end ultimately make a unified model.”
Suhail Doshi: Midjourney is currently the best image generation model
“We, you know, we have, everyone has to acknowledge that Midjourney is very good. You know, they're, they are the best at this thing. We would, I would happily, I'm happy to admit that.”
Suhail Doshi: AI builders should benchmark against market leaders, not outdated baselines
“I think it's incumbent on us to try to compare ourselves to the thing that's best, even if we lose. Even if we're not the best. Right. And you know, at some point, if we are able to surpass my journey, then we, you know, we only have ourselves to compare ourse…”
Doshi: Fréchet Inception Distance Is a Bad Metric for Image Models
“Fit is generally a bad metric. You know, it's good up to a point, and then it kind of like is irrelevant.”
Doshi: Playground Users Generate Millions of Images Every Day
“One benefit of Playground is that we have users making millions of images every single day, and so we can just ask them.”
Doshi: Playground AI Plans to Publish an Open Evaluation Benchmark Next Year
“Hopefully next year, I think we will try to publish kind of like a benchmark that anyone could use. That we evaluate ourselves on and that other people can, that we think does a good job of approximating real world performance because we've tried it and done i…”
Doshi: Ideogram Bests DALL-E at Text Synthesis in Image Models
“Ideogram has done a good job of maybe the best job. Dolly kind of has like a, it has like a hit rate.”
Doshi: Playground v2 kept SDXL's exact architecture for tool compatibility
“We kept the playground V two are exactly the same as SD Excel, not because not out of laziness, but just because we wanted, we knew that the community already had tools.”
Doshi: Playground filters NSFW images from training data to ease runtime filtering
“Yeah, I mean, we personally, we filter out NSFW type of images in our dataset, so that it's, you know, so our safety filter stuff doesn't have to work as hard.”
Doshi Claims Playground AI Was First to Ship LCM Preview Feature
“I think we were the first company to integrate it... I think we were the first company to actually ship a quick LCM thing.”
Doshi: LoRA Stable Diffusion Creator Simu Works With Playground AI
“The person that brought Laura's to stable diffusion actually works with us on, on some projects. His name is Simu”
Doshi: AI inference DevOps is similar to scaling high-volume API servers
“I don't find, I find the DevOps for inference to be relatively easy. It doesn't feel that different than, you know, I think we had thousands and thousands of servers at Mixpanel just for dealing with the API had such huge quantities of volume that I didn't fin…”
Doshi: AI training infrastructure software is nascent and broken
“You know, if you have, like, a node that goes down, then your, you know, training run crashes, and then you have to somehow be resilient to that, and I would say training infra software is very early, feels very broken. Feel, I can tell in 10 years it would be…”