Ruiz: GPT-4V turns low-quality photos of paper wireframes into functional websites
Steve Ruiz · The Accidental AI Canvas - with Steve Ruiz of tldraw · Jan 5, 2024 · at 41:41
Steve Ruiz, founder of tldraw, shows how GPT-4V can ingest rough photos of physical paper wireframes to generate functioning HTML web applications.
0:00 / 0:56exact quote · 56.8s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“I can just take the screenshot here of, you know, that the dude posted of the drawing that he had made, you know, it's not even like a good photo. There's a pen, you know, across one of the screens, et cetera. But if you just give that with no other information to, like, as a prompt you get back a pretty good result. Like, it loading right now, but I've done this before, and yeah, like, you get back you know, just from this, like, photo of a piece of paper on the guy's desk, you know, you have a like, Not, not completely arbitrary result, like working website here that was inferred from just that picture, with no other input, not, not even like titles or anything else, and of course it's like responsive and all this stuff.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Steve Ruiz
Ruiz: Drawing apps combined with generative models have no competitive moat
“There's nothing really defensible about like, Hey, it's an, it's a drawing app plus an LCM like model because there's going to be a lot of those models and there's going to be a lot of drawing apps.”
Ruiz: Collaborative whiteboards and text-to-diagram AI will become ubiquitous software commodities
“Kanban boards are in every productivity app now. I think the same thing is going to happen with collaborative whiteboards. It's like people like them. I'm making it easy. People are already doing it even without Teal Draw when it's hard. Like, yeah, that's goi…”
Ruiz: Visual software challenges stem from aesthetic taste, not technical complexity
“It wasn't necessarily like technical problems that were really hard. It was more subjective problems where I think the thing that was lacking was the taste or the opinions or the, like the feeling for what good solutions were.”
Ruiz: Users will never get complex AI prompts right on the first attempt
“You probably won't get ever get the prompt just right. Even if you have like a really, really, really good, you know, Three generations from now agent. Like you still have to put that information in. But you're never going to put the, all of the information in…”
Ruiz: Interactive prototypes are necessary to define novel interactive software features
“In all those places, you kind of have to build something in order to figure out what you want to build.”
Ruiz: Infinite canvas tools must follow standardized interaction conventions
“Like, if you're making a canvas this way, you have to kind of do them all. Like, your undo, redo should work in a specific way. Your selection should work in a specific way. Like, you know, the camera position and how the camera moves should work in a, you kno…”