Course Description
Without any question, machine learning and related methods are part of our daily life. Ranging from recommender systems in online shops, over smartphone apps to self-driving cars, we constantly come across high-level use cases of very sophisticated machine learning techniques. At the same time, more and more disruptive achievements in medicine, natural science and industry become possible due to machine learning. Hence, we all are more than aware that a profound understanding of this field is of tremendous importance.
This course is an introduction to machine learning with the objective to provide a first clear mathematical and scientific understanding of the field. It targets students with an interest in a mathematically oriented introduction to the field. Those students have been exposed to initial linear algebra and calculus courses. Also, a first course on probability should have been attended, in which the concept of univariate random variables has been established. Given this knowledge, this course provides a completely self-contained mathematical oriented introduction into the field of machine and statistical learning. The introduction covers everything from moderately sophisticated applications and profound modeling (multivariate random variables, statistical decision theory, prediction error estimation, bias-variance decomposition, etc.), over a majority of the existing standard methods (kNN, linear, polynomial, kernel regression; linear discriminant analysis, logistic regression; k-means clustering, principle component analysis; ridge regression; multilayer perceptrons, simple deep neural networks), including algorithms with many examples that are backed by example online codes in Jupyter notebooks, up to mathematical proofs. At the same time, all content is introduced in a structured way with definitions, examples, lemmas, theorems and proofs, as known from mathematics courses.
Overall, this course enables students at an early stage to gain a detailed mathematical and scientific understanding of the developed methods. Thereby, this course answers entry questions like “Why to use machine learning?” or “How to solve a given task?”, but also tries to approach the truly tricky ones like “How does it work?”, “Why does it work?” and maybe even “Why doesn’t it work?”.
Literature
- Trevor Hastie, Robert Tibshirani and Jerome Friedman: “The Elements of Statistical Learning” (Second Edition), Springer
- Shai Shalev-Shwartz and Shai Ben-David: “Understanding Machine Learning: From Theory to Algorithms”, Cambridge University Press
- Aurelien Géron: Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow
- Ovidiu Calin: “Deep Learning Architectures: A Mathematical Approach”, Springer, 2020
Syllabus
- Primer in probability
- Introduction to traditional learning
- Statistical Decision Theory
- Linear regression
- (Stochastic) Gradient Descent
- Estimation of prediction error
- Bias-variance tradeoff
- Linear classification
- Unsupervised learning
- Advanced regression
- Artificial neural networks
- Deep learning