Herzig: Large language models are not designed for predictive enterprise tasks
Philipp Herzig · SAP: Bringing the ‘Operating System’ of a Company into the AI Era with CTO Philipp Herzig · Apr 23, 2026 · at 26:43
Philipp Herzig, CTO of SAP, explains why LLMs cannot easily replace classical machine learning for enterprise tabular predictions.
0:00 / 0:22exact quote · 23.0s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“Now, if you want to do these predictions, quite frankly, then the challenge is large language models are not made for this, right? The way how they, you know, generate just one token after another, essentially, in a sequence-to-sequence modeling. I mean, They're language models, right? And they do this phenomenally well. But if you still want to do these predictors where you have to go back to these classical machine learning approaches, right?”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Philipp Herzig
Herzig: Enterprise AI adoption gap is widening rather than narrowing
“We also see that the AI adoption in the enterprise is still not where we want to see it, right? Like there's this Gartner curve, right? Where say like there's this AI innovation race, and then there's this AI outcome race, right? Then the gap almost increases,…”
Herzig: Agentic coding succeeds because outcomes are automatically verifiable
“The problem of why does agent decoding work so well, Sarah, is of course you can verify the outcome, right? You can either say, hey, is the program compiling or are your unit tests, right? Does it work, et cetera.”
Herzig: Developers never actually adopted test-driven development in practice
“The reality is nobody did it. At least I never did it because it was so much more fun. It was not very popular at the end.”
Herzig: Most enterprise agents will rely on tool calling, not GUI
“I still believe for the most part it will, The majority will live with tool calling, right? And agents running in the background and so on, right? Because you also don't, you know, maybe want to have the browser open all the time. Okay, we can do this with hea…”
Herzig: The era of software requiring manual user intelligence is over
“The time is clearly over where you design software, where the dump software, where the, that requires the intelligence to sit in front of the computer.”
Herzig: System of Record Data Is Insufficient for Autonomous Enterprise Agents
“If you just look into the system of record today, that data is insufficient for this grand vision that everybody has, that it becomes this autonomous enterprise or like the agency of these agents is increasing right over time.”