Jan 10, 2025 · 46m · saastr
Adding AI to SaaS: Inside the AI Product Strategies of Figma, Cloudflare, GitHub and Ramp
Mario Rodriguez · 10m spoken Diego Zaks · 9m spoken Vincent van der Meulen · 8m spoken Dane Knecht · 8m spoken Dani Grant · 4m spoken
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this panel discussion, engineering and product leaders from GitHub, Ramp, Cloudflare, and Figma share practical strategies for building and scaling production AI tools. They explore adaptive roadmapping, specialized evaluation frameworks, organizational talent models, and the transition toward seamless, invisible AI embedded directly into user workflows.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. How this is scored →
Jason as informed peer 1.3 Guest teaching 1.4 Guest disagreement 0.2 Jason pushing back 0.4
speaking balance: gold is Jason, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 32:57 Diego rejects corporate risk aversion
Diego Zaks strongly criticizes conventional corporate habits where managers delay releases for months attempting to avoid mistakes, arguing that fearless velocity is the only survival strategy.
Hardest push from Jason ▶ 27:26 Dani demands concrete evaluation implementation detailsDani Grant interrupts high-level talk to drill into how offline evaluation suites and test suites are structured in practice.
Biggest teaching moment ▶ 26:18 Mario breaks down compiler offline evaluations versus real-world failureMario Rodriguez explains to the room why relying solely on 95 percent offline benchmark scores leads to false confidence, teaching how real user evals diverge from lab metrics.
Jason holds their own ▶ 29:18 Dani contrasts deterministic QA with non-deterministic AI evalsDani Grant frames the evaluation question with deep technical clarity, contrasting traditional solved deterministic software testing against probabilistic AI quality bars.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Jason as informed peer | Guest teaching | Guest disagreement | Jason pushing back | Why |
|---|---|---|---|---|---|---|
| Mario Rodriguez on Building GitHub Copilot | 0 | 0 | 0 | 0 | Solo presentation segment where Mario Rodriguez details the origin story of GitHub Copilot and focus on latency and prompt engineering. The host does not participate during the talk, keeping host-side dynamic scores at zero. | |
| Diego Zaks on Ramp's Invisible AI | 0 | 0 | 0 | 0 | Diego Zaks delivers a monologue explaining Ramp's philosophy of making financial workflows disappear rather than adding visible chat interfaces. There is zero host intervention or friction. | |
| Dane Knecht on Cloudflare's Edge AI | 0 | 0 | 0 | 0 | Dane Knecht presents Cloudflare's edge GPU infrastructure strategy in a monologue format. Host interaction is limited to the introductory handoff. | |
| Vincent van der Meulen on Figma AI | 0 | 0 | 0 | 0 | Vincent van der Meulen outlines Figma's AI feature bundle and internal evaluation processes without interruption. The dynamic is purely informative and collaborative. | |
| Adapting Strategic Roadmaps to Rapid AI Shifts | 2 | 3 | 0 | 0 | Dani opens the panel discussion by asking how teams maintain roadmaps amidst unpredictable AI progress. Mario explains GitHub's cone-of-confidence strategic bets and monthly reviews. | |
| Three Planning Horizons and Experimental Culture | 0 | 0 | 0 | 0 | The panelists share their respective planning horizons and hackathon cultures across Cloudflare, Ramp, and Figma. The host remains silent throughout the panelist-to-panelist handoffs. | |
| Small Demo Teams and Offline AI Evals | 3 | 4 | 0 | 2 | Dani presses Dane and Mario on the tactical, day-to-day reality of running experiments and evaluations. Mario educates on compiler offline evaluations (coffee) and the limitations of synthetic benchmarks. | |
| Figma's Canvas-Based Visual Evaluation Tooling | 4 | 3 | 2 | 1 | Dani observes that non-deterministic AI breaks standard software testing playbooks and asks about time-to-market trade-offs. Diego and Dane argue forcefully that speed and high bet velocity beat perfectionist planning. | |
| Building and Reskilling Internal AI Teams | 2 | 2 | 0 | 1 | Dani asks about team composition and internal reskilling programs for applied AI. Mario and Vincent explain that enthusiasm and organic product engineering matter more than formal corporate reskilling. | |
| Operational AI and Multidisciplinary AI Integration | 1 | 2 | 0 | 0 | Vincent, Diego, and Dane highlight internal operational AI applications in support and risk operations. The dialogue is collaborative and instructional without pushback. | |
| Five-Year Future of AI in Software | 2 | 1 | 0 | 0 | Dani poses a closing prompt regarding software transformation over a five-year horizon. The panelists deliver their visionary closing thoughts before the session concludes. |
Statements from this episode (26)
Dani Grant: GitHub Copilot Has 1.8M Paying Users Across 77,000 Companies
“Copilot has 1.8 million paying users from 77,000 companies.”
GitHub CPO: Copilot Engineered for 100-150ms Global Latency
“We worked out really hard in making the latency be a hundred milliseconds, a 150 milliseconds, no matter where you were in the world.”
GitHub CPO: Next-Gen Copilot Aims to Turn Non-Programmers Into Creators
“What I'm really excited about the next generation of Copaldo is transforming into this product that Your grandmother, your child, or you yourself, even if you're not a programmer, can actually interact and create something overall.”
Dani Grant: Ramp Reached $300M ARR in About Three Years
“They reached three hundred million of ARR in about three years.”
Dani Grant: Ramp Saves 25,000 Businesses $1B and 10M Minutes With AI
“25,000 businesses use RAMP. They, and they use AI to save those 25,000 businesses about a billion dollars and ten million minutes”
Ramp VP: Platform Turns Predicted Repeated Actions Into Keyboard Shortcuts
“Once you do something on RAMP a couple times, we essentially predict the next action And instead of clicking around, we just make it a keyboard command and just hit enter and just get through the list.”
Dani Grant: One-Third of Fortune 500 Companies Use Cloudflare
“A third of the Fortune 500 uses them.”
Cloudflare SVP: 170 Data Centers Retrofitted With GPUs in One Year
“So yeah, in the past year we've retrofitted our 300 cities, 500, over 500 data centers with GPUs. Over a 170 of them have them today and pushing out more bigger, faster inference every day.”
Cloudflare SVP: AI Is Most Powerful Embedded Invisibly, Not as Chatbots
“AI can become really powerful when you're building into your application, not as a chat interface, but really as something that disappears and becomes part of it.”
Figma Designer: AI Features Only Ship After Passing Perfect Output Evals
“So for every feature that we built, we set up these little tests. Where we have the expected perfect outputs and we compare, we then compare that expected perfect outputs to the AI outputs. And we do that for every change. And only when the expected outputs is…”
GitHub CPO: Product Predictability Is Impossible With Unpredictable LLMs
“In AI, you cannot have predictability. LLMs are not predictable even on that.”
Cloudflare SVP: AI Product Teams Must Review Roadmaps Every 4-6 Weeks
“Really, if you're not reviewing the roadmap and really asking yourself is the next thing that we're building the right thing you're probably waiting too long.”
Ramp VP: Planning AI Around Current Model Limits Guarantees Failure
“Rather than trying to figure it out, decide today with the limitations that we have today, which in the world of AI, that It's like a complete, that just leads to complete failure within weeks.”
Cloudflare SVP: Early AI Failures Often Reflect Model Limits, Not Bad Ideas
“You need to invest in, and create a place where it's okay to fail. And cause the failure might be that, it might not be a failure of the idea. It might be a failure of the models that aren't there yet. And you need to wait for this technology to catch up.”
GitHub CPO: AI Experimentation Is Useless Without Over-Investing in Offline Evals
“At GitHub we invest in this thing we call coffee, which is compiler offline evaluation, and that's one area that I have found to be a place that you have to over invest in AI if you will, if you need to experiment, because you could experiment all day in AI an…”
GitHub CPO: 95% Offline Evaluation Scores Still Yield Bad AI Products
“Even if you get an offline 95, by the way, usually that means the product, when it gets to market, it's a really bad product, at least in AI world, in my opinion.”
GitHub CPO: Every Existing Public AI Benchmark Can Be Gamed
“I don't like benchmarks out there by the way, because you could game every single one of them. In my opinion, but what I do like about benchmarking is that it gives you a view into a set of scenarios that then you could then figure out, are you getting better …”
Figma Evaluates AI Search by Rendering Results Directly Onto the Canvas
“So for instance, for our search feature, we would have the AI search model, just spit out all the search results on the Figma infinite canvas. And then we could go in with a Figma plugin to actually rate those search results as good or bad. And we would do tha…”
Figma Has Approximately 200 AI Plugins in Its Ecosystem
“We have something like 200 AI Figma plugins right now.”
GitHub CPO: AI Products Require AI Engineers Alongside Applied Scientists
“The right thing is not to say you need a bunch of applied science. I think you also need to grow the company to be AI engineers plus applied science. And that's what at the end might create the right product overall. Instead of just investing in the deep tech.”
GitHub CPO: Copilot Development Is Decentralized Across All Product Teams
“What I would say is we try to not centralize it. There shouldn't be just one co-pilot team. There's a co-pilot intelligence platform team, and that's true, but I would say the repos team is a co-pilot team. The PR team is a co-pilot team. The issues team is a …”
Figma Engineers Reskill in AI Organically Through Personal Side Projects
“What we'll see is that a lot of these people reskill automatically because they're so passionate about AI and are just constantly building side projects in their own time. We don't even need to do the reskilling, reskilling.”
Ramp VP: AI's Biggest Transformation Has Been Internal-Facing Operations
“AI can be customer facing, but in our, at least in my experience, it's been mostly internal facing. That's where the biggest transformation has come.”
Cloudflare Evaluates Every Web Request Using an AI Security Model
“And then our product teams for the products that we actually shipped to our customers, we've been using AI and ML for years. Every single request that comes through us goes through a security model that decides is the request good or bad.”
GitHub CPO: Platform Will Become Completely AI-Native in Five Years
“What I would say is it looks more AI native. One of the beauties of what we're living through right now is that these LLMs can actually understand natural language in a way that none of the other tools before could. So I think if we really lean into that and g…”
Figma Designer: AI Will Collapse Design and Engineering Roles Within 5 Years
“Yeah, five years from now, nobody of course knows, but I do think some level of role collapse is going to happen, which is super exciting. So designers will be able to create, software engineers will be able to create designs and all of these roles are going t…”