Statistical Machine Learning

Moodle: Most communication for this class will take place on Moodle. Please sign up here .
Lecture: Wednesdays, 13:30 - 16:00 o'clock, ZOOM for now, S103/226 when again possible
Exercises: Wednesdays, 16:15 - 17:00 o'clock, ZOOM for now, S103/226 when again possible
Language: English

Lectures and exercises take place via Zoom. Please visit the Moodle page to find the link.

Motivation Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it. Many researchers also think it is the best way to make progress towards human-level AI. In this class, you will learn about statistical machine learning techniques, and gain some practice implementing them and getting them to work for yourself.



Lectures

This course consists of a mix of online lectures and flipped classroom sessions. That is, aside from our online lectures, you may also want to watch the aditional videos provided, even before the lecture. Please prepare a list of 5 questions as extra homework for those videos, since we (all of you) may also try to answer your questions in class. We will also present our own work. We may even ask you to read papers, let's see. Aside from the “5-questions” homework, there are also four written assignments involving both theory and programming. Handing in those assignments regularly will earn you a bonus for the final exam.

This is the syllabus for the Summer 2020 iteration of the course.

l
Lecture # Date Description Our Slides Videos
1 Introduction April 22, 2020 Course Introduction, Basic ML Intoduction .pdf Intro, T. Mitchell, CMU
Intro, A. Ihler, UCI
2 Linear Algebra April 29, 2020 Vector, matrix, (Pseudo) Inverse, Eigenvectors, ... .pdf Matrix Algebra for Engineers, J. Chasnov, Udacity/HKUST
Linear Algera Basics, C. Pryby, Udacity
Eigenvalues and Eigenvectors, MIT 18.06C Linear Algebra
3 Statistics May 6, 2020 Random Variables, Distributions, Moments, ... .pdf Probabilitiy distributions and their Properties, B. Carlson
Random Var. & prob. dist., M. Rooduijn, E. van Loon, UoA
Random Variables, Expection, Covariance, ..., IIT, India
4 Optimization May 13, 2020 Convex, (Un)constraint Optimization, Lagragian Multiplier, Duality, Gradient descent, Newton, CG, ... .pdf Convex Probelems, T. Balch, A. Chakraborty, Georgia Tech
Convexity, S. Dasgupta, UCSD
Convexity and Optimization, A. Smola, CMU
Duality, S. Dasgupta, UCSD
Lagrangian Multipliers, G.J. Gordon, CMU
5 Bayesian Decision Theory May 20, 2020 Class Conditional Probs, Class Prios, Bayesian Probs, Decision Boundary, Risk Minimization , ... .pdf Empirical and Expected Risk, Smola, UC Berkeley
Risk and Loss Functions, Dini, Pravahan, AT&T
Risk Minimization, Andy Park, Purdue
6 Probabilisty Density Estimation May 27, 2020 Maximum Likelihood Estimation Bayesian Estimation, Nonparametric Density, Curse of Dimensionality, Expectation Maximization, ... .pdf Estimating Probabilities from Data, Weinberger, Cornell
MLE Gaussian, Littman, Isbell, Kolhe, GeorgiaTech
Curse of Dimesionality, Littman, Isbell, Kolhe, GeorgiaTech
Mixture of Gaussians, Fox, U. Washington
Expectation Maximization, Ng, Stanford
kNN and Parzen Window, Lavrenko, U. Edingburgh
Hyper Cube, Dini, Pravahan, AT&T



Assignments and Exercises

Aside from the “5-questions” homework, there are also four written assignments involving both theory and programming. Handing in those assignments regularly will earn you a bonus for the final exam. Pleses visit Moodle for further information, and for submitting them.




Further Material and Links