{"id":206,"date":"2020-11-16T19:51:33","date_gmt":"2020-11-16T19:51:33","guid":{"rendered":"http:\/\/www.peter-zaspel.de\/?p=206"},"modified":"2021-09-19T21:05:55","modified_gmt":"2021-09-19T21:05:55","slug":"machine-learning-spring-2020","status":"publish","type":"page","link":"https:\/\/www.peter-zaspel.de\/?page_id=206","title":{"rendered":"Machine Learning (Spring 2020)"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">Overview<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Official course description<\/h4>\n\n\n\n<p>Machine learning (ML) is about algorithms which are fed with (large quantities of) real-world data, and which return a compressed &#8220;model&#8221; of the data. An example is the &#8220;world model&#8221; of a robot: the input data are sensor data streams, from which the robot learns a model of its environment &#8212; needed, for instance, for navigation. Another example is a spoken language model: the input data are speech recordings, from which ML methods build a model of spoken English &#8212; useful, for instance, in automated speech recognition systems. There exist many formalisms in which such models can be cast, and an equally large diversity of learning algorithms. However, there is a relatively small number of fundamental challenges which are common to all of these formalisms and algorithms. The lecture introduces such fundamental concepts and illustrates them with a choice of elementary model formalisms (linear classifiers and regressors, radial basis function networks, clustering, neural networks, hidden Markov models). Furthermore, the lecture also provides a refresher of required mathematical material from probability theory and linear algebra.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Literature<\/h4>\n\n\n\n<p><strong>Primary text<\/strong>:<br>Hastie, Tibshirani, Friedman: \u201cThe Elements of Statistical Learning\u201d (Second Edition), Springer<\/p>\n\n\n\n<p><strong>Recommended reading<\/strong>:<br>Shalev-Shwartz, Ben-David: \u201cUnderstanding Machine Learning: From Theory to Algorithms\u201d, Cambridge University Press<\/p>\n\n\n\n<p><strong>Further useful references for the math background:<br><\/strong>Linear algebra and probability reviews available at <a href=\"http:\/\/cs229.stanford.edu\/syllabus.html\">http:\/\/cs229.stanford.edu\/syllabus.html<\/a>&nbsp;<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Grading<\/h4>\n\n\n\n<p>The grades for this lecture will be determined as follows:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>final exam (100 %)<\/li><\/ul>\n\n\n\n<p>There will be no other formal requirements.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Final exam<\/h4>\n\n\n\n<p>All rules, times, etc. are consolidated in the <a href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6027\">Final exam announcement<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lecture style, tutorials, homeworks and further information<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Online teaching<\/h4>\n\n\n\n<p>Online classes are carried out as follows:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><strong>Video recordings of the class content<\/strong> that would have been normally presented in the lecture slot. The videos can be either watched via the embedded player or downloaded by clicking on the name of the video.<\/li><li>Online quizzes that can be carried out and repeated at any time.<\/li><li>The slides that were uploaded for each lecture anyway.<\/li><li><strong>Questions &amp; Answer Video-Conferencing sessions<\/strong><\/li><\/ol>\n\n\n\n<p>The Video conferencing sessions take place in Microsoft Teams in the respective course on<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Wednesdays, starting at 9:00<\/li><li>Thursdays, starting at 17:30<\/li><\/ul>\n\n\n\n<p>The instructor will keep the meeting running for at least ten minutes. If no student shows up in these ten minutes, the meeting is stopped.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Online tutorials<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li>Offered via video conferencing in MS Teams in the &#8220;Team&#8221; of this lecture<\/li><li>Weekly tutorial classes offered by TAs:<ol><li>Mondays, 15:45-17:00<\/li><li>Wednesdays, 17:15-18:30<\/li><\/ol><\/li><li>Content:<ul><li>Repetition and discussion of lecture content<\/li><li>Discussion of upcoming and graded homework<\/li><\/ul><\/li><li>No mandatory attendance.<br>\u2192 attendance highly recommended in order to be successful<\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Homeworks<\/h4>\n\n\n\n<p><strong>Flavour<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>one assignment sheet per week, published on moodle<\/li><li>contents of each assignment sheet:<ul><li>\u223c3 tasks: theory (manually computing predictor, proving, &#8230;)<\/li><li>\u223c1 task: programming (implementing ML algorithms)<br>\u2192 programming language C\/C++<\/li><\/ul><\/li><\/ul>\n\n\n\n<p><strong>Submission<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>weekly deadline: Friday, 12:00 (noon)<\/li><li>submission format:<ul><li>theory: via moodle<\/li><li>programming: via moodle<\/li><\/ul><\/li><li>submissions in groups of 1 \u2013 3<br>\u2192 depending on class size and homework participation<br>\u2192 might be subject to adjustments<\/li><\/ul>\n\n\n\n<p><strong>Grading (of non-mandatory homeworks)<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>exercises graded with points by TAs<\/li><li>The points that students receive for their homework will have no influence on the final grade, i.e. doing the exercises is not mandatory.<\/li><li>However: Students that are not able to achieve at least 50% of the points from the exercises should expect that they have not got enough training in the content and therefore will most likely have issues in the final exam.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Code demos<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kNN_regression.ipynb\">kNN regression<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kNN_classification.ipynb\">kNN classification<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=linear_regression_1d.ipynb\">Linear regression (on 1d input)<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=linear_regression_2d.ipynb\">Linear regression (on 2d input)<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=bivariate_gaussian.ipynb\">Bivariate Gaussian density<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=linear_regression_z_score_test.ipynb\">Z score<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=training_error.ipynb\">Training error<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kNN_prediction_error_validation_set_approach.ipynb\">Generalization error approximation by validation set approach<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kNN_prediction_error_method_comparison.ipynb\">Generalization error approximation by different methods for synthetic data<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kNN_prediction_error_method_comparison_cancer.ipynb\">Generalization error approximation by different methods for cancer data<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kNN_regression_bias_variance_tradeoff.ipynb\">Bias &#8211; Variance tradeoff<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=classification_linear_vs_nonlinear.ipynb\">Linear vs. non-linear classification<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=linear_classification.ipynb\">Linear vs. quadratic discriminant analysis<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kMeans_clustering.ipynb\">kMeans clustering<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=PCA_compression.ipynb\">PCA compression<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=PCA_synthesis.ipynb\">PCA synthesis<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=basis_expansions_quadratic.ipynb\">Regression using quadratic polynomial basis expansion (1D)<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=basis_expansion_quadratic_2d.ipynb\">Regression using quadratic polynomial basis expansion (2D)<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=kernel_regression.ipynb\">Regression using kernel model<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=sgd_linear_regression.ipynb\">Gradient Decent methods applied in linear regression<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=nn_simple_keras_regression.ipynb\">Neural Network regression in Keras<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=nn_simple_keras_regression_initialization.ipynb\">Neural Network weight initialization<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=nn_simple_keras_regression_batch_normalization.ipynb\">Neural Network batch normalization<\/a><\/li><li><a href=\"https:\/\/mybinder.org\/v2\/gh\/zaspel\/teaching.git\/master?filepath=nn_from_mlp_to_cnn.ipynb\">Neural Network image classification with MLP and CNN URL<\/a><\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Lecture content<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">Content until March 12 (i.e. in-person teaching)<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">Lecture slides of February 5<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4731\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4731\">of February 6<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4773\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4773\">of February 12<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4778\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4778\">of February 13<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4809\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4809\">of February 19<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4841\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4841\">of February 20<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4855\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4855\">of February 26<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4908\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4908\">of February 27<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4915\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4915\">of March 4 (repetition class)<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4923\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4923\">of March 5 (second part of repetition)<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4924\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4924\">of March 5<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4956\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4956\">of March 11<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4983\">Lecture <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4689\">slides<\/a><\/a> <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4983\">of March 12<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for March 18<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4982\">Lecture slides (part 1\/3 and 2\/3)<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5084\">Lecture slides (part 3\/3)<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4987\">Introduction to Bias vs. Variance<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=4988\">Proof of Bias-Variance decomposition<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5086\">Bias-Variance decomposition for kNN regression<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for March 19<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5087\">Lecture slides (part 1\/6 and 2\/6)<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5111\">Lecture slides (part 3,4,5,6 of 6)<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5088\">Proof of Bias-Variance dec. for kNN regression<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5089\">Demo for Bias-Variance Tradeoff in kNN regression<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5113\">Introduction to estimation of prediction error<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5114\">What is training error?<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5115\">Training error by example<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5117\">Why training error is still important<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for March 25<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5217\">Lecture slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5224\">What is generalization error?<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5225\">Expected generalization and the strange T<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5228\">Introduction to empirical error estimation<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for March 26<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5288\">Lecture slides (part 1\/4)<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5305\">Lecture slides (part 2,3,4 of 4)<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5289\">More advanced generalization error estimators<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5302\">Introduction to classification<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5303\">Linear methods in classification<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5304\">Classification by linear regression<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 1<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5387\">Lecture slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5388\">Introduction into Linear Discriminant Analysis<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5389\">The theorem behind LDA<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5390\">Proving the theorem behind LDA<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5391\">The LDA algorithm<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5392\">How to measure prediction error &amp; further classification methods<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 2<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5500\">Lecture slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5502\">Introduction to unsupervised learning<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5503\">Introduction to clustering<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5504\">Towards efficient combinatorial clustering<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5505\">Proof of the loss reformulation<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5506\">The K-means clustering algorithm<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5507\">Examples<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 15<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5561\">Lecture slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5563\">Introduction to PCA &amp; Compression motivation<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5565\">Compression motivation cont.<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5566\">Compression example<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5567\">Data predictor motivation<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5568\">Data predictor motivation cont.<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5569\">Data predictor &#8211; Simple example<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 16<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5562\">Lecture slides (part 1,2,3\/7)<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5618\">Lecture slides (part 4,5,6,7\/7)<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5570\">Data predictor &#8211; Digits example<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5571\">How to compute PCA<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5572\">PCA algorithms<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5612\">Building more complex models by basis expansions<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5613\">Least squares regression for basis expansion models<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5614\">Quadratic polynomial model<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5615\">General polynomial models<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 22<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5619\">Lecture slides (part 1,2\/5) <\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5736\">Lecture slides (part 3,4,5\/5)<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5616\">Kernel-based models<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5617\">Least squares regression for kernel-based models<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5733\">Motivation for Ridge Regression<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5734\">Theory of Ridge Regression<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5735\">Kernel Ridge Regression<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 23<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5784\">Lecture slides for April 23 File<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5778\">Introduction to Neural Networks and the multilayer perceptron<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5779\">Definition and graphical representation of the MLP<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5782\">Activations and how to do regression with the MLP<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5783\">How to do classification with the MLP<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 29<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5847\">Lecture Slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5843\">Some important remarks on MLPs<\/a><\/li><li>Video:<a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5845\">Our first &#8220;deep&#8221; network<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5844\">Introduction to Gradient Decent methods<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5846\">Understanding Gradient Decent File<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for April 30<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6003\">Lecture slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5848\">Stochastic Gradient Decent<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5849\">Mini-batch Gradient Decent and some remarks<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5986\">Example of using Gradient Decent for Linear Regression<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for May 6<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6002\">Lecture slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5988\">Introduction towards backpropagation<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5994\">Forward propagation<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5997\">A central statement on how to compute gradient entries<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5998\">The backpropagation formula<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for May 7<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6001\">Lecture slides (parts 1,2 \/ 5)<\/a><\/li><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6088\">Lecture slides (parts 3,4,5 \/ 5)<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=5999\">Proof of the backpropagation formula<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6000\">The backpropagation algorithm<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6078\">Training an FFNN<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6079\">The vanishing \/ exploding gradients problem<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6080\">The batch normalization layer (1)<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Material for May 13<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li><a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6091\">Lecture slides<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6086\">The batch normalization layer (2)<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6087\">Overfitting in FFNN training<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6090\">Motivation for CNNs<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6120\">The convolutional layer<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6093\">The pooling layer and CNN architectures<\/a><\/li><li>Video: <a class=\"\" href=\"https:\/\/moodle.jacobs-university.de\/mod\/resource\/view.php?id=6119\">Image classification by MLP and CNN<\/a><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Final exam discussion lecture (May 14)<\/h4>\n\n\n\n<p>The final lecture will be again a live lecture without lecture recording. It will take place in the original lecture slot.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview Official course description Machine learning (ML) is about algorithms which are fed with (large quantities of) real-world data, and which return a compressed &#8220;model&#8221; of the data. An example is the &#8220;world model&#8221; of a robot: the input data<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","_mc_calendar":[],"footnotes":""},"class_list":["post-206","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=\/wp\/v2\/pages\/206","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=206"}],"version-history":[{"count":4,"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=\/wp\/v2\/pages\/206\/revisions"}],"predecessor-version":[{"id":211,"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=\/wp\/v2\/pages\/206\/revisions\/211"}],"wp:attachment":[{"href":"https:\/\/www.peter-zaspel.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=206"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}