Campbell: Blindly prompting LLMs to debug code leads to nonsensical rabbit holes
Duncan Campbell · AI for climate: a real world test · Jun 15, 2023 · at 28:54
Duncan Campbell recounts testing ChatGPT's ability to debug battery dispatch optimization code without developer oversight.
0:00 / 0:13exact quote · 14.0s
720p mp4 · rendered on demand · StarZero watermark
“In debugging, if you kind of just, like, keep prompting it, like, fix this, here's the error, like, here's the message I'm getting back in error, and you just kind of, like, assume it's gonna be smart. You can wind up in, like, rabbit holes that just don't make any sense at all.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Duncan Campbell
Campbell: AI will not replace software engineers but make individuals team-scale
“I don't see this replacing software engineers. Just as it empowered me, like I very much see a great software engineer just using this to save a ton of time, right? Telling it exactly what to do with a level of specificity about programming that I could never …”
Campbell: Building complex software with ChatGPT requires iterative prompting
“It's very hard to kind of, like, top-down build it, because there's a bunch of little errors everywhere. It's, like, hard to track what's going on. And I pretty quickly reverted to, like, bottom-up building it. Like, building one element, like, slowly working …”
Campbell: ChatGPT operates like a fast junior engineer needing domain guidance
“ChatGTP is not like a ten-year industry veteran with deep domain expertise, right? It's a decent sort of like Junior software engineer, and its benefit is that it's very fast, right? But you can't just sort of like let it take the reins and run with the proble…”
Campbell: ChatGPT's 'permaflow' eliminates human context-switching penalties
“Every time you go back to you know, chat.openai.com and go into this chat, It has a perfect state of where it was previously, right? So I said four or five hours of continuous work, but in reality, it was a bunch of 45 minute segments. And I didn't have to lik…”
Campbell: LLMs will empower Excel and systems thinkers across the energy industry
“Energy industry is full of people who are like great at Excel and really good at systems thinking, but are not programmers. So I think you could see a lot of people empowered by this.”
Campbell: Parsing utility tariffs is a holy grail LLM energy use case
“A persistent issue in the electricity industry for as long as I've been doing this job is utility tariffs, right? Every utility has some 700 page document that outlines massive The complex nuances of how users will pay for electricity. And we're talking about …”