Deep inversion – towards adaptive physical inversion models in imaging

Carola-Bibiane Schönlieb works on the algorithms which make it possible to extract information reliably and efficiently from imaging data

Imaging and inverse problems are ubiquitous in modern technology and applications such as biomedical imaging, remote sensing and ‘big data’ applications such as Google image search, to name just a few. The applications are based on the ability to extract information from imaging data in a reliable and efficient manner. Mathematics is at the core of the solution to these problems, involving questions about the modelling, analysis and computational realisation of such methods based on partial differential equations (PDEs), variational models and machine learning.

My research aims to push the boundaries of what can be extracted from imaging data, providing robust and efficient algorithms based on PDEs and variational models for computing high-resolution information from large- and high dimensional imaging data. These mathematical insights will enable the development of new imaging tools for medical doctors to look inside the human body and aid diagnosis and treatment decisions, or for forest conservators to monitor forest health from automated analyses of airborne imaging data. Further applications will be in the development of automated image search, and virtual restoration algorithms for digitised museum archives. 

Figure 1. Size matters: A geometric graph clustering and scale detection approach for automated computation of the size of blazes on the forehead of pied-flycatcher birds. Image courtesy of Luca Calatroni

Variational models and PDEs constitute deterministic and mathematically rigorous image analysis and inversion models with stability and approximation guarantees as well as a control on the qualitative and physical properties of the solution (e.g. smoothness). This feature is important, especially when the answer the model gives will decide whether an autonomous car should turn left or right, whether tissue in a CT (computed tomography) scan is cancerous or normal, or if the umbrella-like object in a suitcase is a potential threat. On the negative side, these methods are rigid in a sense that they can be adapted to data only to a certain extent. 

Having worked with researchers in a variety of areas of application, I appreciate the importance of data-adaptiveness and mathematical imaging methodologies that are bespoke to a particular application. In my recent research I have pursued the development of mathematical imaging methods that are rigorous but nonetheless adaptive to the imaging data at hand, bringing ideas from machine learning such as bilevel optimisation and deep neural networks to the framework of variational models and PDEs. 

I would like to acknowledge all her fantastic collaborators without whom this work and its prospect would not have been possible. Special thanks for the model-based learning in imaging work go to Juan Carlos De Los Reyes, Ozan Öktem and Simon Arridge.

Professor Carola-Bibiane Schönlieb
University of Cambridge
Philip Leverhulme Prize