Mathew Lodge

CEO, Diffblue · 1 appearance on the record.

computed by AI from the episodes · how this works → · full disclaimer →

Mathew Lodge is the CEO of Diffblue, an AI platform utilizing reinforcement learning to autonomously test software. He discusses topics surrounding the AI for code landscape and prompt engineering.

12statements → 6claims → 2claims resolved → 3.92/5average certainty → 2.33/5average debate potential → 4.1/5argument clarity · the sources →

0 supported 2 partly supported 0 contradicted 4 not checkable as stated how the 6 claims stand · each chip opens the sources

2 predictions · 4 assertions · 2 opinions · 3 insights · 1 disclosure · every statement was checked. The predictions and assertions are the 6 claims: statements the public record can support or contradict. 2 are resolved, and 4 name no date, number or outcome precise enough to check. Everything else (opinions, insights, what ifs, disclosures) can never be settled by the record, so it carries no assessment.

The record, in short

What the tape says about how Mathew argues and how the claims held up. Everything they said, and everything said about them, is in the tabs below.

Argument clarity: do they answer the question? how? →

4.1 / 5 directness 4 · coherence 4.5 · precision 3.8 · compression 3.6

redirected or did not address 1 of 8 assessed questions (13%). Watch them ▸

This is a score against a rubric. It is not a rank. Every host question → answer exchange is scored with names hidden on directness, coherence, precision and compression, 1–5 each, on meaning alone: disfluencies are ignored, and only raw unedited episodes count. This is the score that measures thought. Every scored exchange, scores shown → · The rubric and its checks →

How they sound: speaking style how? →

257 words/min while actually speaking · 36 um and uh per 1k words

Measured by listening to the audio itself: 6,196 words across 1 episode of raw-level tape, transcribed verbatim with every um and uh kept, each one attributed only where the alignment onto our timed stream is unambiguous. These are measurements of speaking style. We do not rank them: across this corpus, fluency and argument quality are nearly uncorrelated (ρ≈0.2), and smooth talking does not signal clear thinking. How it's measured →

What they said12 Episodes1

Everything Mathew Lodge said on the MAD Podcast that made the record, most notable first. Filter by type, assessment or year in the ledger →

Opinion

Mathew Lodge argues prompt engineering is a fallacy and merely trial-and-error

“And so the idea that there's something called prompt engineering, engineering in the sense of you're applying an approach because it's going to give you the output that you desire. Is a complete fallacy. At best it's trial and error. It's prompt trial and erro…”

Mathew Lodge Oct 25, 2023 ▶ 34:49 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Prediction Not checkable as stated

Lodge: Open-source AI will ultimately defeat proprietary models

“I think so. I think because it allows that level of innovation I just talked about, and I think also it gives people the ability to specialize these models to a particular task.”

Mathew Lodge Oct 25, 2023 ▶ 38:42 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Assertion Not checkable as stated

Diffblue generated 3,100 unit tests in eight hours for a bank

“You know, for one of the banks, they had an application. We wrote like 3100 tests in about eight hours. Now they estimated that that's like a year's work for a single developer, 3000 tests. And we can do that in eight hours.”

Mathew Lodge Oct 25, 2023 ▶ 21:33 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Assertion Partly supported

Private equity mainframe software acquisitions and price hikes drove companies to Java

“Because PE companies bought all of those software products and raised the price.”

Mathew Lodge Oct 25, 2023 ▶ 25:19 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Insight

Interactive AI code suggestions are strictly bottlenecked by human review speed

“You could really still only go as fast as a human can go, because a human has to review that, decide whether it's useful or not, if it is useful, make it, fit it into the code and finish the work. You're still basically going at the speed of a human, whereas w…”

Mathew Lodge Oct 25, 2023 ▶ 32:44 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Assertion Not checkable as stated

Diffblue generates unit tests in 1.5 seconds versus 40 seconds for LLMs

“We built a version of our product where we use the large language model to generate all the tests. So we took our reinforcement lending engine out and put in large language model, and we did a lot of work on prompts and so on. And you know, our product will wr…”

Mathew Lodge Oct 25, 2023 ▶ 36:13 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Opinion

Lodge: Brexit caused an exodus of UK tech talent

“We had, you know, we had a Brexit exodus of talent where people just said, you know what, I'm just going, I'm going back to France or I'm going back to, you know, wherever.”

Mathew Lodge Oct 25, 2023 ▶ 42:22 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Insight

Mathew Lodge: Developers dislike unit tests because incentives favor feature delivery

“Fundamentally they're not paid to write tests. They're paid to deliver the functionality, the new things in the application that fixes the application, the updates and tests are just there to, as part of the process to help them.”

Mathew Lodge Oct 25, 2023 ▶ 16:49 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Insight

Lodge: AI code generation mirrors the shift from assembly to compiled languages

“The analogy I like to make is, it's kind of like, ah, the shift, you know, 30, ah, 35 years ago from sort of assembly language to compiled languages, right? So what you did as a developer really changed”

Mathew Lodge Oct 25, 2023 ▶ 31:30 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Prediction Not checkable as stated

The AI market will shift toward smaller models running without GPUs

“It's one of the reasons I think that we're going to see more smaller models and you've got startups now emerging, which are essentially trying to, like, distill down, like from a large model into something much smaller that will run without a GPU, and they're …”

Mathew Lodge Oct 25, 2023 ▶ 37:02 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Assertion Partly supported

Lodge: DeepMind shaved 7% off C++ sort algorithm using reinforcement learning

“Google DeepMind has also applied this approach Do things like code optimization, where essentially they're searching for more efficient implementations of a particular algorithm. So they shaved about seven percent off the sort algorithm in the C++ library, whi…”

Mathew Lodge Oct 25, 2023 ▶ 7:37 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Disclosure

Lodge: Diffblue uses ensemble of reinforcement learning and LLMs

“Yeah, we, so we use an ensemble approach. So we've got different aspects of the models and we've obviously been playing with large language models to see if we can improve on that process. So some of this is like, how do you make predictions? And you can use l…”

Mathew Lodge Oct 25, 2023 ▶ 8:36 Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself

Appearances (1)

Episode	Date	Speaking time
Diffblue’s AI Testing Paradigm Shift — CEO Mathew Lodge Explains How Code Writes Itself	Oct 25, 2023	29m

Mathew Lodge

The record, in short

Argument clarity: do they answer the question? how? →

How they sound: speaking style how? →

Appearances (1)

Turn any episode into a week of clips.