Course Description
Data-driven modelling typically starts with the collection of large amounts of data from multiple sources. These sources can be as diverse as sensors, server traffic, simulation data, imaging, etc. This data is then used either in models that rely entirely on the given data or together with simulations, i.e. computational models, for example to solve inverse problems. Many of these models are used in everyday applications and provide significant improvements in science, industry and business. Given their growing impact in practice, it is important to ask to what extent we can trust these models.
This is the starting point of this course. We introduce the Bayesian formalism as an approach to estimating uncertainty in inference tasks. Inference can appear in supervised machine learning models, where we are given data that is subject to uncertainty and we want to build models that give us information about the uncertainty associated with the predictions. Inference can also appear in inverse problems (medical imaging, climate reconstruction, etc.), where we know a given process only through a forward model that describes how a (hidden) parameter of interest translates into observable information, and we want to quantify this parameter, including statements about its uncertainty. Together we develop the necessary mathematical language, some theory and many practical algorithms to understand and develop applications of supervised machine learning and inverse problems where we are able not only to compute reliable estimates but also to provide uncertainty information for these predictions. We are also familiarizing ourselves with a probabilistic programming language that allows us to tackle more complex hierarchical and graphical models, as well as Bayesian neural networks. In summary, we are going to teach machine learning models how to say “I’m not sure!” or even “I don’t know!”.
This course builds on the previous Mathematical Machine Learning course. It allows for a deeper understanding of machine learning mechanisms, but also complements applications in scientific computing where properties of numerical models are to be inferred from real data. The Mathematical Machine Learning module provides the necessary (advanced) background in supervised machine learning techniques, while typical probability classes as well as classes on calculus and linear algebra as known from the Computer Science undergraduate program provide the necessary good working knowledge in mathematics. The course aims to discuss the topic itself in a self-contained way, so that all important notions are (re)introduced. While we will discuss and sometimes prove the necessary theory, a major focus will be on the practical use of the algorithms introduced and developed, which will be intensively covered in the exercises.
Literature
- Christopher M. Bishop: “Pattern Recongnition and Machine Learning”, Springer.
- Sebastian Reich and Colin Cotter: “Probabilistic Forecasting and Bayesian Data Assimilation”, Cambridge University Press
- Kevin P. Murphy: “Probabilistic Learning – An Introduction”, MIT Press
Syllabus
- Review of probability theory
- Bayesian inference
- Point estimates
- Bayesian regression
- Closed-form results for Gaussians
- Model analysis and uncertainty
- Approximative sampling methods
- An inverse problem
- Model comparison and selection
- Probabilistic programming
- Bayesian neural networks