Mathew Lodge argues prompt engineering is a fallacy and merely trial-and-error
Mathew Lodge · Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself · Oct 25, 2023 · at 34:49
Mathew Lodge is the CEO of Diffblue. He responds to industry commentary around prompt engineering for large language models.
0:00 / 0:17exact quote · 17.2s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And so the idea that there's something called prompt engineering, engineering in the sense of you're applying an approach because it's going to give you the output that you desire. Is a complete fallacy. At best it's trial and error. It's prompt trial and error.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Mathew Lodge
Lodge: Open-source AI will ultimately defeat proprietary models
“I think so. I think because it allows that level of innovation I just talked about, and I think also it gives people the ability to specialize these models to a particular task.”
Diffblue generated 3,100 unit tests in eight hours for a bank
“You know, for one of the banks, they had an application. We wrote like 3100 tests in about eight hours. Now they estimated that that's like a year's work for a single developer, 3000 tests. And we can do that in eight hours.”
Private equity mainframe software acquisitions and price hikes drove companies to Java
“Because PE companies bought all of those software products and raised the price.”
Interactive AI code suggestions are strictly bottlenecked by human review speed
“You could really still only go as fast as a human can go, because a human has to review that, decide whether it's useful or not, if it is useful, make it, fit it into the code and finish the work. You're still basically going at the speed of a human, whereas w…”
Diffblue generates unit tests in 1.5 seconds versus 40 seconds for LLMs
“We built a version of our product where we use the large language model to generate all the tests. So we took our reinforcement lending engine out and put in large language model, and we did a lot of work on prompts and so on. And you know, our product will wr…”