Schönlieb: Photoshop's Content-Aware Fill is based on mathematical research
Carola Schönlieb · Mathematical Approaches to Image Processing with Carola Schönlieb · Y Combinator · May 9, 2018 · at 4:13
Applied mathematician Carola Schönlieb discusses the mathematical roots of digital inpainting and image restoration with Craig Cannon on the Y Combinator Podcast.
0:00 / 0:15exact quote · 15.1s
▶ Watch the full episode on YouTube → 720p mp4 · rendered on demand · StarZero watermark
“And I mean, also the content-aware fill is actually very much based on some of the things that have been initiated by people like Andrea Batozzi. So, I mean, the technique is Different in what Photoshop is using, but it's still based on research in mathematics, in fact.”
quote is from the automated transcript, cleaned for reading: filler sounds and stutters are removed, nothing is rephrased. names can be misheard (the analysis reads context, assessments check outside sources). how →
More from Carola Schönlieb
Medical AI models trained on one scanner brand fail on others
“Also the type of scanner you're using, are you using a G or a Siemens or Toshiba or whatever they have different settings, And they have different ways of going from the measurements to an image. And so, you know, if you train an algorithm, for instance, a neu…”
Schönlieb: Deep neural networks outperform handcrafted methods in image denoising
“Image denoising nowadays, I think the best image denoising approaches are actually coming from deep neural networks. So, you know, these handcrafted methods get more and more beaten in terms of performance. By some of these neural network approaches.”
Schönlieb: Exactly minimizing training loss often hurts neural network generalization
“You do not necessarily need to solve your optimization problem, your training exactly. And maybe sometimes, or most of the time you actually don't want it, want to save it exactly because you only have a finite amount of training examples. And so when you thin…”
Schönlieb: ML super-resolution creates probable images but cannot verify accuracy
“Maybe, you know, if you have all these machine learning methods which have learned to look at just pixels and then know what is a very probable match in terms of high resolution, maybe at some point you can do it, but then you don't know how, if you're right o…”
Schönlieb: Handcrafted image models provide mathematical proofs and error estimates
“We can prove properties about the denoising abilities of these methods of how stable they are, for instance, to perturbations in the images. We know how that works, so we can prove things about that. We have error estimates and things like this.”
Schönlieb: CT reconstruction is constrained by limits on patient radiation exposure
“You want a very high resolution image because you want to look at all the details in the body. But you don't want to measure so many line integrals because you don't want to radiate the patient so much. You don't want to send tons of x-rays through, through th…”