Official Course Description
Machine learning (ML) is about algorithms which are fed with (large quantities of) real-world data, and which return a compressed “model” of the data. An example is the “world model” of a robot: the input data are sensor data streams, from which the robot learns a model of its environment — needed, for instance, for navigation. Another example is a spoken language model: the input data are speech recordings, from which ML methods build a model of spoken English — useful, for instance, in automated speech recognition systems. There exist many formalisms in which such models can be cast, and an equally large diversity of learning algorithms. However, there is a relatively small number of fundamental challenges which are common to all of these formalisms and algorithms. The lecture introduces such fundamental concepts and illustrates them with a choice of elementary model formalisms (linear classifiers and regressors, radial basis function networks, clustering, online adaptive filters, neural networks, or hidden Markov models). Furthermore, the lecture also (re-) introduces required mathematical material from probability theory and linear algebra.
- Hastie, Tibshirani, Friedman: “The Elements of Statistical Learning” (Second Edition), Springer
- Shalev-Shwartz, Ben-David: “Understanding Machine Learning: From Theory to Algorithms”, Cambridge University Press
- Géron: Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow
Further useful references for the math background:
Linear algebra and probability reviews available at http://cs229.stanford.edu/syllabus.html
Grading and final exam
The grades for this lecture will be determined by a final written exam (100%). If not otherwise indicated, it will cover all content of the course.
The grade of the final exam can be improved by 0.33 grade points if the following two conditions are both fulfilled:
- at least 50% of the points from the weekly quizzes of the live lecture are achieved, and
- at least 50% of the points from the weekly homework assignments are achieved.
There is no re-scheduling, repetition or compensation option for the Bonus Achievements in case of illness. This holds even if a sick certificate is presented.
All official announcements for this course will be posted via the below announcement forum, to for which all students have a mandatory subscription. Additionally, there is a forum for interaction with the instructor and the TA wrt. general questions. All course-related (non-private) communication should be placed to that general discussion forum.
Course related questions and inquiries
Culture of interaction
- Please feel free to ask questions at any time!
- Please tell the instructor, if he is too slow / fast / boring / excited / . . . by giving feedback via the lecture forum or in a private communication.
- Please use the “virtual” open-door policy, i.e. send a brief message and we can have a meeting.
Officially, this course has two lecture slots per week, namely Tuesdays, 11:15-12:30 and Fridays 8:15-9:30. The initial (online!) meeting of the course will take place on Tuesday, February 2, 11:15-12:30 in this time period. You can access the meeting via the link further down this page.
Afterwards, however, we will chose a more flexible blended learning approach. Specifically, the course will be taught in a partially synchronous (i.e. live), partially asynchronous, and partially personal way. What this exactly means is discussed below.
Lecture notes – “Delivery” of factual content
During the semester, the instructor will newly develop lecture notes, i.e. pretty much a book on machine learning with examples and explanations. It will be a typesetted document that will finally contain all lecture contents that are required to be known to pass the exam. Every week, students are required to read a new section of these lecture notes. Thereby they will passively acquire the theoretical knowledge of the lecture. Certainly, acquiring knowledge just by reading is a tedious task. Therefore, this “core content delivery” is enriched by several additional offers.
Live lecture – Reminding, strengthening, exploring
Within the Tuesday lecture slot, more precisely on Tuesdays, 11:15-12:30 using the link below, a live (for now) online meeting (on Zoom) takes place. During that meeting, a “camera on” policy holds, i.e. all students are supposed to switch on their cameras.
The Tuesday meetings will have the following components:
- Quiz: All meetings will start by a Moodle quiz for which the students have 10 minutes of time to complete it. The quiz is an open-book quiz, i.e. students are allowed to use the lecture notes or whatever other information source that could help them to solve the quiz. The quiz is also one of the two mandatory components of the Bonus Achievements.
- Strengthening: Depending on the content, additional examples, re-iteration of theory, quick paper and pencil tasks etc. (partially in small groups) are offered by the instructor in order to dive deeper into the topic.
- Exploring: Depending on the content, students might actively work with some machine learning code to try out the just acquired knowledge.
- Q & A: Students will have the opportunity to ask questions. Plenty of time, will be available for this part, if students desire it.
“Bookable” slot – Personalized content support
During the the Friday slot, i.e. on Fridays starting from 8:15, a single student or a group of students can “book” slots of 10 minutes (maximum 2 slots, if in groups) with the instructor for a more personalized support with the lecture content. With this offer, the instructor wants to overcome the issue of sometimes strongly varying learning habits of students, which is in particular pronounced in large groups. Use the below scheduling tool to book a slot and the below zoom link to enter the meeting.
Per week, four tasks are published as homework assignments on Moodle. They will support the student in further practicing the content that was read and discussed in the previous week. Also, they are the second component of the Bonus Achievements. About three of the tasks cover theory work (manually computing a predictor, proving, …) and about one task covers the implementation of machine learning algorithms in Python or C/C++.
- deadline: weekly on Tuesdays, 10:00 AM
- format: PDF only, via Moodle, at the bottom of this page
- in groups: 1-3 students, depending on class size, participation, etc. (we might have to adapt this depending on the number of submissions)
- Groups are defined via the Submission group assignments Database activity below. Follow the instructions given with the activity.
- Only one person per group submits the homework. All other will get the points assigned.
- exercises evaluated with points by TAs
- at least 50% of the points are need as one of the requirements of the Bonus Achievements
- Reference solutions
- TAs provide graded homeworks and tutorials discuss solutions.
- Reference solutions will be published one week after the content has been discussed in the tutorials.
- no points are given for copied homework (affects both parties)
Once per week, a TA offers a tutorial. In the tutorial, students can ask further questions. Moreover, the TAs discuss the solutions to the previous assignments and answer questions on the new assignments. The tutorial takes place on Thursdays, at 19:15 in an online format.
Univariate probability (Recap)
Task until Tuesday, February 9:
- Read chapter 0 of the lecture notes.
Live lecture on Tuesday, February 9
- Quiz on univariate probability (only available on Moodle)
- Lecture tasks on univariate probability
- Notes from the lecture on univariate probability (only available on Moodle)
Homework assignments until February 16, 10:00 AM
Task until Tuesday, February 16:
- Read chapter 1 of the lecture notes.
Live lecture on Tuesday, February 16
- Quiz on multivariate probability (only available on Moodle)
- Lecture tasks on multivariate probability
- Notes from the lecture on multivariate probability (only available on Moodle)
Homework assignments until February 23, 10:00 AM
Introduction to machine / statistical learning
Task until Tuesday, February 23:
- Read chapter 2 of the lecture notes.
Live lecture on Tuesday, February 23
- Quiz on introduction to ML (only available on Moodle)
- Lecture tasks on Introduction to ML
- Empty Jupyer notebook
- Molecular data set example (only available on Moodle)
- Solution notebook for Iris data set task (only available on Moodle)
- Notes from the lecture on introduction to ML (only available on Moodle)
Homework assignments until March 2, 10:00 AM
Statistical decision theory
Task until Tuesday, March 2:
- Read chapter 3 of the lecture notes.
Live lecture on Tuesday, March 2
- Quiz on statistical decision theory (only available on Moodle)
- Lecture tasks for statistical decision theory
- kNN classification example
- Solution notebook for kNN classification on Iris data set (only available on Moodle)
Homework assignments until March 9, 10:00 AM
Task until Tuesday, March 9:
- Read chapter 4 of the lecture notes.
Live lecture on Tuesday, March 9
- Quiz on linear regression (only available on Moodle)
- Lecture tasks for linear regression
- Notes from the live lecture on linear regression (only available on Moodle)
- Notebook for the linear regression task (only available on Moodle)
Homework assignments until March 16, 10:00 AM
(Stochastic) Gradient descent
Task until Tuesday, March 16:
- Read chapter 5 of the lecture notes.
Live lecture on Tuesday, March 16
- Quiz on (Stochastic) Gradient Descent (only available on Moodle)
- Lecture tasks for (stochastic) gradient descent
- Notes from the live lecture on (stochastic) gradient descent (only available on Moodle)
- Notebook with non-linear model (only available on Moodle)
Homework assignments until March 23, 10:00 AM
Estimation of prediction error
Task until Tuesday, March 23:
- Read chapter 6 of the lecture notes.
Live lecture on Tuesday, March 23
- Quiz on Estimation of prediction error (only available on Moodle)
- Lecture tasks for estimation of prediction error
- Notebook with training & generalization error estimation (only available on Moodle)
Homework assignments until April 6, 10:00 AM