Figma Designer: AI Features Only Ship After Passing Perfect Output Evals
Vincent van der Meulen · Adding AI to SaaS: Inside the AI Product Strategies of Figma, Cloudflare, GitHub and Ramp · Jan 10, 2025 · at 16:40
Vincent van der Meulen, AI lead at Figma, explains how Figma manages non-deterministic AI quality through structured evaluations before public release.
0:00 / 0:22exact quote · 22.5s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“So for every feature that we built, we set up these little tests. Where we have the expected perfect outputs and we compare, we then compare that expected perfect outputs to the AI outputs. And we do that for every change. And only when the expected outputs is close enough, only when the AI output is close enough to the expected output, that's when we actually put these features into the wild.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Vincent van der Meulen
Figma Designer: AI Will Collapse Design and Engineering Roles Within 5 Years
“Yeah, five years from now, nobody of course knows, but I do think some level of role collapse is going to happen, which is super exciting. So designers will be able to create, software engineers will be able to create designs and all of these roles are going t…”
Figma Engineers Reskill in AI Organically Through Personal Side Projects
“What we'll see is that a lot of these people reskill automatically because they're so passionate about AI and are just constantly building side projects in their own time. We don't even need to do the reskilling, reskilling.”
Figma Evaluates AI Search by Rendering Results Directly Onto the Canvas
“So for instance, for our search feature, we would have the AI search model, just spit out all the search results on the Figma infinite canvas. And then we could go in with a Figma plugin to actually rate those search results as good or bad. And we would do tha…”