##### Official Course Description

Machine learning (ML) is about algorithms which are fed with (large quantities of) real-world data, and which return a compressed “model” of the data. An example is the “world model” of a robot: the input data are sensor data streams, from which the robot learns a model of its environment — needed, for instance, for navigation. Another example is a spoken language model: the input data are speech recordings, from which ML methods build a model of spoken English — useful, for instance, in automated speech recognition systems. There exist many formalisms in which such models can be cast, and an equally large diversity of learning algorithms. However, there is a relatively small number of fundamental challenges which are common to all of these formalisms and algorithms. The lecture introduces such fundamental concepts and illustrates them with a choice of elementary model formalisms (linear classifiers and regressors, radial basis function networks, clustering, online adaptive filters, neural networks, or hidden Markov models). Furthermore, the lecture also (re-) introduces required mathematical material from probability theory and linear algebra.

##### Course literature

- Hastie, Tibshirani, Friedman: “The Elements of Statistical Learning” (Second Edition), Springer
- Shalev-Shwartz, Ben-David: “Understanding Machine Learning: From Theory to Algorithms”, Cambridge University Press
- Géron: Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow

**Further useful references for the math background:**

Linear algebra and probability reviews available at http://cs229.stanford.edu/syllabus.html

##### Grading and final exam

The grades for this lecture will be determined by a final written exam (100%). If not otherwise indicated, it will cover all content of the course.

**Bonus Achievements**

The grade of the final exam can be improved by 0.33 grade points if the following two conditions are **both** fulfilled:

- at least 50% of the points from the weekly quizzes of the live lecture are achieved,
**and** - at least 50% of the points from the weekly homework assignments are achieved.

There is no re-scheduling, repetition or compensation option for the Bonus Achievements in case of illness. This holds even if a sick certificate is presented.

##### Course communication

All official announcements for this course will be posted via the below announcement forum, to for which all students have a mandatory subscription. Additionally, there is a forum for interaction with the instructor and the TA wrt. general questions. All course-related (non-private) communication should be placed to that general discussion forum.

Announcements

Course related questions and inquiries

##### Culture of interaction

- Please feel free to ask questions at any time!
- Please tell the instructor, if he is too slow / fast / boring / excited / . . . by giving feedback via the lecture forum or in a private communication.
- Please use the “virtual” open-door policy, i.e. send a brief message and we can have a meeting.

### Course Setup

Officially, this course has two lecture slots per week, namely Tuesdays, 11:15-12:30 and Fridays 8:15-9:30.** The initial (online!) meeting of the course will take place on Tuesday, February 2, 11:15-12:30 in this time period.** You can access the meeting via the link further down this page.

Afterwards, however, we will chose a more flexible blended learning approach. Specifically, the course will be taught in a partially synchronous (i.e. live), partially asynchronous, and partially personal way. What this exactly means is discussed below.

##### Lecture notes – “Delivery” of factual content

During the semester, the instructor will newly develop lecture notes, i.e. pretty much a book on machine learning with examples and explanations. It will be a typesetted document that will finally contain all lecture contents that are required to be known to pass the exam. Every week, students are required to read a new section of these lecture notes. Thereby they will passively acquire the theoretical knowledge of the lecture. Certainly, acquiring knowledge just by reading is a tedious task. Therefore, this “core content delivery” is enriched by several additional offers.

Forum to collect typos / mistakes

##### Live lecture – Reminding, strengthening, exploring

Within the Tuesday lecture slot, more precisely on **Tuesdays, 11:15-12:30 **using the link below, a *live* (for now) online meeting (on Zoom) takes place. During that meeting, a **“camera on” policy** holds, i.e. all students are supposed to switch on their cameras.

The Tuesday meetings will have the following components:

All meetings will start by a Moodle quiz for which the students have 10 minutes of time to complete it. The quiz is an open-book quiz, i.e. students are allowed to use the lecture notes or whatever other information source that could help them to solve the quiz. The quiz is also one of the two mandatory components of the Bonus Achievements.**Quiz:****Strengthening:**Depending on the content, additional examples, re-iteration of theory, quick paper and pencil tasks etc. (partially in small groups) are offered by the instructor in order to dive deeper into the topic.**Exploring:**Depending on the content, students might actively work with some machine learning code to try out the just acquired knowledge.**Q & A:**Students will have the opportunity to ask questions. Plenty of time, will be available for this part, if students desire it.

##### “Bookable” slot – Personalized content support

During the the Friday slot, i.e. on Fridays starting from 8:15, a single student or a group of students can “book” slots of 10 minutes (maximum 2 slots, if in groups) with the instructor for a more personalized support with the lecture content. With this offer, the instructor wants to overcome the issue of sometimes strongly varying learning habits of students, which is in particular pronounced in large groups. Use the below scheduling tool to book a slot and the below zoom link to enter the meeting.

Scheduler for “bookable slots”

Bookable slots meeting (online) URL

##### Homework assignments

Per week, four tasks are published as homework assignments on Moodle. They will support the student in further practicing the content that was read and discussed in the previous week. Also, they are the second component of the Bonus Achievements. About three of the tasks cover theory work (manually computing a predictor, proving, …) and about one task covers the implementation of machine learning algorithms in Python or C/C++.

**Submission**- deadline: weekly on Tuesdays, 10:00 AM
- format: PDF only, via Moodle, at the bottom of this page
- in groups: 1-3 students, depending on class size, participation, etc. (we might have to adapt this depending on the number of submissions)
- Groups are defined via the Submission group assignments Database activity below. Follow the instructions given with the activity.
- Only one person per group submits the homework. All other will get the points assigned.

**Evaluation**- exercises evaluated with points by TAs
- at least 50% of the points are need as one of the requirements of the Bonus Achievements

**Reference solutions**- TAs provide graded homeworks and tutorials discuss solutions.
- Reference solutions will be published one week after the content has been discussed in the tutorials.

**Cheating**- no points are given for copied homework (affects both parties)

Forum to find submission partner

##### Tutorials

Once per week, a TA offers a tutorial. In the tutorial, students can ask further questions. Moreover, the TAs discuss the solutions to the previous assignments and answer questions on the new assignments. The tutorial takes place on **Thursdays, at 19:15** in an online format.

### Univariate probability (Recap)

**Task until Tuesday, February 9:**

- Read chapter 0 of the lecture notes.

**Live lecture on Tuesday, February 9**

- Quiz on univariate probability (only available on Moodle)
- Lecture tasks on univariate probability
- Notes from the lecture on univariate probability (only available on Moodle)

**Homework assignments until February 16, 10:00 AM**

### Multivariate probability

**Task until Tuesday, February 16:**

- Read chapter 1 of the lecture notes.

**Live lecture on Tuesday, February 16**

- Quiz on multivariate probability (only available on Moodle)
- Lecture tasks on multivariate probability
- Notes from the lecture on multivariate probability (only available on Moodle)

**Homework assignments until February 23, 10:00 AM**

### Introduction to machine / statistical learning

**Task until Tuesday, February 23:**

- Read chapter 2 of the lecture notes.

**Live lecture on Tuesday, February 23**

- Quiz on introduction to ML (only available on Moodle)
- Lecture tasks on Introduction to ML
- Empty Jupyer notebook
- Molecular data set example (only available on Moodle)
- Solution notebook for Iris data set task (only available on Moodle)
- Notes from the lecture on introduction to ML (only available on Moodle)

**Homework assignments until March 2, 10:00 AM**

### Statistical decision theory

**Task until Tuesday, March 2:**

- Read chapter 3 of the lecture notes.

**Live lecture on Tuesday, March 2**

- Quiz on statistical decision theory (only available on Moodle)
- Lecture tasks for statistical decision theory
- kNN classification example
- Solution notebook for kNN classification on Iris data set (only available on Moodle)

**Homework assignments until March 9, 10:00 AM**

### Linear regression

**Task until Tuesday, March 9:**

- Read chapter 4 of the lecture notes.

**Live lecture on Tuesday, March 9**

- Quiz on linear regression (only available on Moodle)
- Lecture tasks for linear regression
- Notes from the live lecture on linear regression (only available on Moodle)
- Notebook for the linear regression task (only available on Moodle)

**Homework assignments until March 16, 10:00 AM**

### (Stochastic) Gradient descent

**Task until Tuesday, March 16:**

- Read chapter 5 of the lecture notes.

**Live lecture on Tuesday, March 16**

- Quiz on (Stochastic) Gradient Descent (only available on Moodle)
- Lecture tasks for (stochastic) gradient descent
- Notes from the live lecture on (stochastic) gradient descent (only available on Moodle)
- Notebook with non-linear model (only available on Moodle)

**Homework assignments until March 23, 10:00 AM**

### Estimation of prediction error

**Task until Tuesday, March 23:**

- Read chapter 6 of the lecture notes.

**Live lecture on Tuesday, March 23**

- Quiz on Estimation of prediction error (only available on Moodle)
- Lecture tasks for estimation of prediction error (only available on Moodle)
- Notebook with training & generalization error estimation (only available on Moodle)

**Homework assignments until April 6, 10:00 AM**

- Assignment Sheet 7 (only available on Moodle)

### Bias – variance tradeoff

**Task until Tuesday, April 6:**

- Read chapter 7 of the lecture notes.

**Live lecture on Tuesday, April 6**

- Quiz on bias-variance trade-off (only available on Moodle)
- Lecture tasks for bias-variance trade-off (only available on Moodle)

**Homework assignments until April 13, 10:00 AM**

- Assignment Sheet 8 (only available on Moodle)

### Classification

**Task until Tuesday, April 13:**

- Read chapter 8 of the lecture notes.

**Live lecture on Tuesday, April 13**Quiz on linear classification (only available on Moodle)

Lecture tasks on linear classification (only available on Moodle)

Notes from the live lecture on linear classification (only available on Moodle)

**Homework assignments until April 20, 10:00 AM**

- Assignment Sheet 9 (only available on Moodle)

### Unsupervised learning

**Task until Tuesday, April 20:**

- Read chapter 9 of the lecture notes.

**Live lecture on Tuesday, April 20**

- Quiz on unsupervised learning (only available on Moodle)
- Lecture tasks on unsupervised learning (only available on Moodle)
- PCA faces synthesis Jupyter notebook (only available on Moodle)

**Homework assignments until April 27, 10:00 AM**

- Assignment Sheet 10 (only available on Moodle)

### Advanced Regression

**Task until Tuesday, April 27:**

- Read chapter 10 of the lecture notes.

**Live lecture on Tuesday, April 27**

- Quiz on advanced regression (only available on Moodle)
- Lecture tasks on advanced regression (only available on Moodle)
- Notes from the live lecture on advanced regression (only available on Moodle)

**Homework assignments until May 4, 10:00 AM**

- Assignment Sheet 11 (only available on Moodle)

### Artificial neural networks

**Task until Tuesday, May 4:**

- Read chapter 11 of the lecture notes.

**Live lecture on Tuesday, May 4**

- Quiz on artificial neural networks (only available on Moodle)
- Lecture tasks on artificial neural networks (only available on Moodle)
- Notes from the live lecture on artificial neural networks (only available on Moodle)

**(Voluntary) Homework assignments until May 11, 10:00 AM**

- Assignment Sheet 12 (only available on Moodle)

### Deep learning

**Task until Tuesday, May 11:**

- Read chapter 12 of the lecture notes.

### Final exam preparation

**Live lecture on Tuesday, May 11, 11:15-12:30**

- Very brief discussion of last content
- Exam preparation
- Q&A

**Personal, non-mandatory offer: Live lecture on Friday, May 15, 8:15-9:30**

- Q&A for any last minute questions, students could come up with