Assertion Supported AI assessment confidence: 95% certainty 4/5 debate potential 2/5

Pedro Domingos: Every major ML algorithm can theoretically learn any function

Pedro Domingos · Fireside Chat: Pedro Domingos, Head of Machine Learning, DE Shaw (FirstMark's Data Driven NYC) · Oct 22, 2019 · at 11:46

Pedro Domingos, machine learning expert and author of The Master Algorithm, explains universal approximation theorems across major machine learning paradigms.

0:00 / 0:06exact quote · 6.8s

▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark

“On a theoretical level, every one of these major machine learning algorithms has a theorem that says if you give it enough data, it can learn any function.”

quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →

The claim, plainly

Every major machine learning algorithm paradigm has a theoretical theorem proving it can learn any function given sufficient data.

a quote alone can be ambiguous, so this restates it with the references and context of the conversation around it resolved. AI-written; the tape below the player is the source.

Assessment

Theoretical machine learning literature establishes universal approximation or universal consistency theorems across all five major paradigms of machine learning algorithms (Connectionists, Analogizers, Symbolists, Bayesians, and Evolutionaries). These theorems prove that with sufficient data and capacity, models within each paradigm can approximate or converge to any target function or probability distribution.

Connectionists: Cybenko (1989) and Hornik et al. (1989) proved universal approximation for feedforward neural networks.
Analogizers: Stone (1977) proved universal consistency for k-nearest neighbors, and Steinwart (2001, 2002) proved universal consistency for SVMs with universal kernels.
Symbolists: Devroye, Györfi, and Lugosi (1996) established universal consistency for decision trees scaling with sample size.
Bayesians: Doob (1949), Schwartz (1965), and Ghosal et al. (2000) proved posterior consistency for nonparametric Bayesian estimators.
Evolutionaries: Cramer (1985), Koza (1992), and Meyerson et al. (2022) established theoretical representation density and universal approximation properties for genetic programming function spaces.

Checked against

[1] wikipedia.org [2] fringeling.com [3] wikipedia.org [4] reddit.com [5] loria.fr [6] medium.com [7] reddit.com [8] wikipedia.org

"support vector machines" universal consistency theorem "symbolic regression" "universal approximation" theorem "decision trees" universal consistency theorem Pedro Domingos five tribes universal approximation theorem universal approximation theorem machine learning paradigms universal approximation genetic algorithms decision trees neural networks SVM "genetic programming" universal approximation theorem universal consistency theorem machine learning algorithms "bayesian" universal consistency theorem "evolutionary algorithm" "universal consistency" OR "universal approximation" "genetic programming" "universal approximation" theorem proof

assessed as of 2026-07-30

This assessment is an AI reading of a public statement, made against the cited references with the confidence shown. It can be wrong. The quote, the tape and the references are all on this page; if something is off, use report an error below. full disclaimer →

Mentioned

machine learning algorithms

⚑ Spot an error? Report it

More from Pedro Domingos

Prediction Not checkable as stated

Pedro Domingos predicts Backprop alone will not reach general AI

“I actually do not believe that that's the case. I think Backprop will not get us there.”

Pedro Domingos Oct 22, 2019 ▶ 14:09 Fireside Chat: Pedro Domingos, Head of Machine Learning, DE Shaw (FirstMark's Data Driven NYC)

Opinion

Domingos: Mandating total model explainability literally makes deep learning illegal

“And at the end of the day, you know, you can't, like the European Union, mandate that every model has to be explainable, right? Because then, if you read that law literally, it makes deep learning illegal, right?”

Pedro Domingos Oct 22, 2019 ▶ 32:31 Fireside Chat: Pedro Domingos, Head of Machine Learning, DE Shaw (FirstMark's Data Driven NYC)

Opinion

Pedro Domingos: Very little of current machine learning capability has been deployed

“With the existing machine learning technology, of what we can do with that, how much have we done? Very little so far. So there's enormous amount for, enormous scope to do things there. Not just in finance, but in a lot of other fields, right?”

Pedro Domingos Oct 22, 2019 ▶ 4:14 Fireside Chat: Pedro Domingos, Head of Machine Learning, DE Shaw (FirstMark's Data Driven NYC)

Prediction Not checkable as stated

Domingos: AI is too early to predict 20 years into the future

“I would say we are so much in the beginning that we can't even really picture where we're going to be 20 years from now.”

Pedro Domingos Oct 22, 2019 ▶ 4:55 Fireside Chat: Pedro Domingos, Head of Machine Learning, DE Shaw (FirstMark's Data Driven NYC)

Insight

Domingos: The hardest problem in AI is learning representation

“The hardest problem in AI is coming up with a representation. Once you've done that, the rest follows, and when you don't do that, you don't really get very far”

Pedro Domingos Oct 22, 2019 ▶ 18:14 Fireside Chat: Pedro Domingos, Head of Machine Learning, DE Shaw (FirstMark's Data Driven NYC)

Insight

Domingos: AI is an amazing tool for authoritarianism

“AI can be a great tool for democracy. It is also unfortunately an amazing tool for authoritarianism.”

Pedro Domingos Oct 22, 2019 ▶ 25:40 Fireside Chat: Pedro Domingos, Head of Machine Learning, DE Shaw (FirstMark's Data Driven NYC)

Pedro Domingos: Every major ML algorithm can theoretically learn any function

More from Pedro Domingos

Turn any episode into a week of clips.