This course consists of a mix of online lectures and flipped classroom sessions. That is, aside from our online lectures,
you may also want to watch the aditional videos provided, even before the lecture. Please prepare a list of 5 questions
as extra homework for those videos, since we (all of you) may also try to answer your questions in class.
We will also present our own work.
We may even ask you to read papers, let's see.
Aside from the “5-questions” homework, there are also four written
assignments involving both theory and programming. Handing in those assignments regularly will earn
you a bonus for the final exam.
This is the syllabus for the Summer 2020 iteration of the course.
| 1 Introduction
|| April 22, 2020
|| Course Introduction, Basic ML Intoduction
|| Intro, T. Mitchell, CMU
Intro, A. Ihler, UCI
| 2 Linear Algebra
|| April 29, 2020
|| Vector, matrix, (Pseudo) Inverse, Eigenvectors, ...
|| Matrix Algebra for Engineers, J. Chasnov, Udacity/HKUST
Linear Algera Basics, C. Pryby, Udacity
Eigenvalues and Eigenvectors, MIT 18.06C Linear Algebra
| 3 Statistics
|| May 6, 2020
|| Random Variables, Distributions, Moments, ...
|| Probabilitiy distributions and their Properties, B. Carlson
Random Var. & prob. dist., M. Rooduijn, E. van Loon, UoA
Random Variables, Expection, Covariance, ..., IIT, India
| 4 Optimization
|| May 13, 2020
|| Convex, (Un)constraint Optimization, Lagragian Multiplier, Duality, Gradient descent, Newton, CG, ...
|| Convex Probelems, T. Balch, A. Chakraborty, Georgia Tech
Convexity, S. Dasgupta, UCSD
Convexity and Optimization, A. Smola, CMU
Duality, S. Dasgupta, UCSD
Lagrangian Multipliers, G.J. Gordon, CMU
| 5 Bayesian Decision Theory
|| May 20, 2020
|| Class Conditional Probs, Class Prios, Bayesian Probs, Decision Boundary, Risk Minimization , ...
Empirical and Expected Risk, Smola, UC Berkeley
Risk and Loss Functions, Dini, Pravahan, AT&T
Risk Minimization, Andy Park, Purdue
| 6 Probabilisty Density Estimation
|| May 27, 2020
|| Maximum Likelihood Estimation Bayesian Estimation, Nonparametric Density, Curse of Dimensionality, Expectation Maximization, ...
Estimating Probabilities from Data, Weinberger, Cornell
MLE Gaussian, Littman, Isbell, Kolhe, GeorgiaTech
Curse of Dimesionality, Littman, Isbell, Kolhe, GeorgiaTech
Mixture of Gaussians, Fox, U. Washington
Expectation Maximization, Ng, Stanford
kNN and Parzen Window, Lavrenko, U. Edingburgh
Hyper Cube, Dini, Pravahan, AT&T
| 7a Expectation Maximization
|| June 10, 2020
|| Extra Slides on Expectation Maximization (EM), ...
k-Means Clustering, Ihler, UCI
Gaussian Mixtures, EM, Ihler, UCI
Gaussian Mixtures, EM, Lavrenko, U. Edinburgh
| 7b Clustering and Evaluation
|| June 10, 2020
|| K-Means, Agglomerative Clustering, Mean Shift, Bias and Variance, ...
Mean Shift, Bobick, Essa, Chakraborty, GeorgiaTech
Overfitting, Ihler, UCI
Bias and Variance , Ihler, UCI
Bias and Variance, Ng, deeplearning.ai
Cross-Validation, Thrun, Udacity
| 8 Regression
|| June 17, 2020
|| Linear Regression, ML Linear Regression, Bayesian Linear Regression ...
Lineare Regression, Ihler, UCI
ML and Linear Regression, de Freitas, UBC
Regularization and Regression, de Freitas, UBC
Bayesian Regression, Lawrence, Sheffield U.
| 9 Classification
|| June 24, 2020
|| Discriminant, Fisher's Linear Discriminant, Perceptron, Logistic Regression ...
Lineare Classifiers: Basics, Ihler, UCI
Lineare Classifiers: Learning Parameters, Ihler, UCI
Intro to Linear Classificaiton, Jensen, TUM
Linear Discriminant Models, Jensen, TUM
Linear Discriminant Analyis, Starmer, StatQuest
Linear Discriminant Analysis, Lavrenko, U. Edinburgh
| 10 Dim. Reduction & ERM
|| July 1, 2020
|| Projection, PCA, Empirical Risk Minimization (ERM), CV Dimension, ...
PCA and SVD, Ihler, UCI
VC Dimension, Ihler, UCI
Generalization Bound, VC Dimension, Huang, Virginia Tech
Approx./Estimation & ERM, Avati, Stanford
| 11 SVM
|| July 8, 2020
|| Hyperplane, Max Margin, Support Vector Machines, Kernels, Slack, ...
Support Vector Classification, Smola, CMU
SVM - Linear SVM/primal form, Ihler, UCI
SVM - Lagrangian and Dual, Ihler, UCI
SVM - Kernels, Ihler, UCI
| 12 Neural Networks
|| July 8, 2020
|| Activation Functions, Neural Networks (NN), Stochastic Gradient, Backpropagation, Convolutional NN, Drop Out, Adam ...
Neural Networks, Ihler, UCI
Backpropagation, Ihler, UCI
Backpropagation, Lavrenko, U. Edinburgh
Convolutional NNs, Soleimany, MIT
Backpropagation, Yeung, Stanford
Convolutional NNs, Yeung, Stanford
| 13 Gaussian Processes
|| July 15, 2020
|| Kernels Revisited, Dual of RBF Networks, Gaussian Processes, Bayesian learning ...
Gaussian Process Regression, Smola, CMU
Introduction to Gaussian Processes, Lawrence, U. Sheffield
Gaussian Processes for Machine Learning, Cunningham, Cambridge
Gaussian Prcoesses, Poupart, U. Waterloo
Introduction to Gaussian Processes, de Freitas, UBC