StarZero All shows Try the podcast clipper →

Sujay Jayakar

Co-founder & Chief Scientist, Convex · 1 appearance on the record.

computed by AI from the episodes · how this works → · full disclaimer →

founderscientistengineerexecutive@sujayakar314 ↗LinkedIn ↗convex.dev ↗

Sujay Jayakar is a co-founder and Chief Scientist at Convex, where he led the development of the Fullstack-Bench evaluation benchmark for coding agents. Previously, he served as a Principal Engineer at Dropbox, where he helped design and rewrite the company's core synchronization engine.

10statements → 7claims → 3claims resolved → 100%fully supported → 3.9/5average certainty → 2.1/5average debate potential → ≈4.0/5argument clarity, estimated →

3 supported 0 partly supported 0 contradicted 4 not checkable as stated how the 7 claims stand · each chip opens the sources

7 assertions · 2 insights · 1 disclosure · every statement was checked. The predictions and assertions are the 7 claims: statements the public record can support or contradict. 3 are resolved, and 4 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.

The record, in short

What the tape says about how Sujay argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.

Their most notable supported claim

Assertion Supported
Cursor Composer solves Convex benchmarks but fails on alternative backends
“We did notice that I mean, with convex, it pretty much autonomously just solves the first two tasks. It has a few round trips on like some errors that are only show up and playing with the front end. And then it's able to complete this files task and kind of g…”
Sujay Jayakar Mar 19, 2025 ▶ 9:36 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex

How they sound: not measured why? →

We measure speaking style by listening to the audio itself, and a fair number needs at least 2,000 words from one person on tape we have measured. There is too little of Sujay Jayakar on measured tape to publish a rate. This says nothing about how they speak.

Everything Sujay Jayakar said on Latent Space that made the record, most notable first. Filter by type, assessment or year in the ledger →

Assertion Supported
Cursor Composer solves Convex benchmarks but fails on alternative backends
“We did notice that I mean, with convex, it pretty much autonomously just solves the first two tasks. It has a few round trips on like some errors that are only show up and playing with the front end. And then it's able to complete this files task and kind of g…”
Sujay Jayakar Mar 19, 2025 ▶ 9:36 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Assertion Not checkable as stated
Claude 3.7 performed worse than Claude 3.5 on Convex evals
“For example, we just tried clod three seven and it performs worse than clod three five on convex evals with the same prompting.”
Sujay Jayakar Mar 19, 2025 ▶ 17:23 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Assertion Supported
AI models struggle debugging Supabase RLS recursion compared to procedural code
“The particular example was like RLS rules and Supabase where debugging like an infinite loop for infinite recursion for the RLS rules was something that the models just really struggled with in a way that we didn't see for procedural code.”
Sujay Jayakar Mar 19, 2025 ▶ 29:57 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Assertion Not checkable as stated
AI hallucinates Convex code due to API similarities with Firebase
“I mean, I think one thing that's kind of interesting given these, you know, models inherent internal structure is that we noticed that a lot of hallucinations come from parts of our API that are like very close to Firebase, but not exactly Firebase.”
Sujay Jayakar Mar 19, 2025 ▶ 22:56 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Assertion Supported
Claude 3.5 in Cursor autonomously codes for 10 plus minutes
“When it has the right feedback in cursor composer, and this was even on cloud three, five, it can just autonomously code for a 10 plus minutes and it can fix its own bugs. It can get to the point where it's like pretty much a fully working app with just an ini…”
Sujay Jayakar Mar 19, 2025 ▶ 3:21 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Assertion Not checkable as stated
OpenAI's o3 outperforms GPT-4o on Convex evals by a small margin
“You know, oh, three does do better than four. Oh, I mean, we use brain trust for tracking all this quantitatively, but I can't remember off the top of my head, but it's not like a slam dunk.”
Sujay Jayakar Mar 19, 2025 ▶ 17:56 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Insight
End-to-end type safety and tight feedback loops improve AI agent performance
“It's like, you know, having just really tight feedback loops and having strong guardrails, like in convex, that's like end to end type safety, but that's like, you know, could take it in many, many different forms, right? Like making it so code is very cheap a…”
Sujay Jayakar Mar 19, 2025 ▶ 29:12 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Insight
Strong library abstractions prevent AI models from breaking backend wiring
“Models, you know, when given the full flexibility of fast API and doing SSE and wiring everything from scratch. It just couldn't help, but messed it up. So having like kind of strong life, like choosing good libraries that have strong abstractions is like, and…”
Sujay Jayakar Mar 19, 2025 ▶ 30:14 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Assertion Not checkable as stated
Jayakar: AI Models Stalled on Convex's Subtle Distinction Between Null and Undefined
“Convex has like a pretty subtle distinction between null and undefined, like just similar to JavaScript. And we noticed that like, because this is a subtle thing that's unexpected, it was The model got stuck on it and couldn't even figure it out. And so this i…”
Sujay Jayakar Mar 19, 2025 ▶ 13:01 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex
Disclosure
Jayakar prototypes AI coding benchmark using Jepsen's Elle model checker
“I think like, you know, I've wired up a version of this, you know, using L, which is the like model checker from Jepson and, you know, can a model write highly correct, highly like code that executes under very high concurrency, even when there's”
Sujay Jayakar Mar 19, 2025 ▶ 18:57 Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex

Appearances (1)

EpisodeDateSpeaking time
Fullstack-Bench: The Eval for Coding Agents — with Sujay Jayakar, Chief Scientist, Convex Mar 19, 2025 21m
Made with StarZero

Turn any episode into a week of clips.

This entire site, over 200 episodes transcribed, diarized, checked and made playable, runs on the StarZero media pipeline. Drop in your own episode and the podcast clipper finds the moments worth sharing, cuts them, captions them, and reframes them for every feed.

Try the podcast clipper Learn more