CMPT 727 G1 Spring 2022
CMPT 727: Statistical Machine Learning. Spring 2022.
Course logistics
- Schedule: Mondays 10:30AM - 11:20AM, Thursdays 10:30AM - 12:20PM.
- Taught remotely.
- Video lectures. Lectures will be added to this Youtube playlist. Last year's lectures.
- Lecture slides
- Zoom link. Passcode = 131950 .
Instructor: Maxwell Libbrecht
- Web page
- Office: TASC 1 9219
- Email: maxwl at sfu dot ca
TA: Heng Liu
- Email: liuhengl at sfu dot ca
- Office Hours: Mondays 10:30AM - 11:30AM Zoom link
Textbooks:
- PMLAI: "Probabilistic Machine Learning: An Introduction" by Kevin Patrick Murphy. Dropbox link to the version of the textbook we're using.
- PMLAT: "Probabilistic Machine Learning: Advanced Topics" by Kevin Patrick Murphy.
- Previous textbook, no longer used: MLPP: "Machine Learning: a Probabilistic Perspective" by Kevin Patrick Murphy.
Groups
Assignment Groups. Last Update Mar 2.
Schedule
Week 1 (Jan 10)
- Reading: PMLAI Ch1.
- Group assignment survey. Due Wed Jan 12 at 12pm Vancouver time.
- Assignment 1. Due Monday, Jan 17, 11:59pm (time zone of your choice). This is a qualifying assignment; based on your performance on the assignment, we will recommend whether you have the prerequisites for the course. (For the purposes of the course grade, everyone will get full credit, so please represent your knowledge accurately.) You should expect to consult reference materials (e.g. textbooks, Wikipedia, etc), but if you find you need to re-read whole textbook chapters, that is a sign that you are missing the prerequisite knowledge.
Week 2 (Jan 17)
- Reading: PMLAI Ch2.1-4 (Probabilistic inference, Bayesian ML). Learn LaTeX in 30 minutes.
- Lectures: Recorded lectures 1-4.2 (see link at top of page). LaTex Tutorial for Beginners
- Assignment 2.
- A2 latex source file
- Special office hour for Latex questions: 10:30AM - 11:30AM Monday, Jan 17, Vancouver time.
Week 3 (Jan 24)
- Reading: PMLAI Ch3.1-3 (Univariate probabilisitic models). PMLAI Ch3.5-6 (Multivariate and linear Gaussian).
- Lectures: Recorded lectures 5-7.
- Assignment 3.
- A3 latex source file.
Week 4 (Jan 31)
- Reading: PMLAI Ch 3.7-8 (Probabilistic graphical models, mixture models). PMLAI Ch 4.1-3 (Parameter estimation).
- Lectures: 8-9.1. (9.1 coming soon.)
- Assignment 4.
- A4 latex source file.
Week 5 (Feb 7)
- Reading: PMLAI 4.4 (Regularization). 5.1-2 (First-order optimization). 5.4-5.5.2 (SGD, constrained optimization).
- Lectures: 10-14.
- Assignment 5.
- A5 latex source file.
Week 6 (Feb 14)
- Reading: PMLAI 5.7 (Bound optimization and EM), 5.8 (Black box optimization), 8.1 (Bayesian decision theory).
- Lectures: 15-18. (Note: As of this writing, Youtube shows an old version of 15.2; the new version should appear soon. The correct version is 26 mins.)
- Assignment 6.
- A6 latex source file.
Reading week: no class (Feb 21).
Week 7 (Feb 28).
- Reading: Ch9.1-9.2.5 (LDA). 9.3-4 (Naive Bayes). Ch10.1-2, 4-5 (Logistic regression).
- Lectures: 19-22.
- Assignment: Midterm programming assignment, due March 14. Instructions Code and data
Week 8 (Mar 7).
- Reading: Ch11.1-3 (linear regression, ridge regression). Ch11.5 (LASSO).
- Lectures: 23-28.
- Assignment: (No written assignment this week.)
Week 9 (Mar 14).
- Reading: Switching to book 2, "Probabilistic Machine Learning: Advanced Topics" (PMLAT). PMLAT Ch4.1-2 (Bayes nets). Ch4.3 (Markov Random Fields).
- Lectures: 29-32.
- Assignment 7.
- A7 latex source file.
Week 10 (Mar 21):
- Reading: PMLAT Ch7 (Inference algorithms overview). Ch9.1-2 (Belief propagation in trees). Ch9.4 (Variable elimination).
- Lectures: 33-34.1.
- Assignment 8.
- A8 latex source file.
Week 11 (March 28):
- Reading: Reading: PLMAT 9.3 (Loopy BP). 9.5 (Junction tree algorithm). 11.1-4 (Rejection sampling).
- Lectures: 35-36.
- Assignment 9.
- A9 latex source file.
Week 12 (April 4):
- Reading: PMLAT 12.1-3 (Gibbs sampling, MCMC). 12.6 (Practical MCMC).
- Lectures: 37-40.
- In-class exercise (nothing to turn in).
Week 13 (April 11):
- Last Zoom lecture (synchronous): Monday April 11, 10:30-11:20 at the usual Zoom link.
- No lectures/reading this week.
- Final programming assignment, due April 25. No late submissions will be accepted, please plan ahead accordingly.
- Instructions Code and data
Syllabus
- Unit 1 (weeks 1-6): Principles of probabilistic learning. PMLAI chapters 1-5, 8.
- Unit 2 (weeks 7-8): Classification: modeling, optimization, regularization. PMLAI chapters 9-11.
- Unit 3 (weeks 9-12): Probabilistic graphical models. MLPP chapters 19-20, 24.
Course information
Unofficial prerequisites: No official prerequisites. However, the course assumes knowledge of machine learning (CMPT 726), probability and linear algebra (MATH 240). The Assignment 1 is a qualifying assignment to show whether you have the necessary background.
The course is open to advanced undergraduates with permission. How to enroll as an undergraduate: Please follow the following steps: (1) Come to the first lecture, where I will explain the expected background. (2) Email me stating that you have taken a course in probability/statistics and a course in machine learning (CMPT 726). If it is appropriate, I will respond with my approval. (3) Fill out the prerequisite waiver form and attach a screenshot of our email exchange.
Lectures: Content lectures will be recorded and posted. We will have one synchronous interactive session per week, Thursdays 10:30am-12:20pm Vancouver time. 10:30-11:20am: Watch this week's lectures synchronously with time for questions. 11:30am-12:20pm: Work on assignment with your group. Attendance is mandatory at the interactive session. If this is challenging for you, e.g. due to time zone differences, please indicate in the survey (coming soon).
Assignments. There will be one written assignment per week, due on Monday. Assignments will be due in two parts. Your first submission is individual, worth 2/3 of the grade. The second submission, due one week later, is a joint group submission, worth 1/3 of the grade. This gives you a chance to fix any mistakes as a group. (Group grade is the maximum of the individual and group submission.)
Groups. The class will be divided into groups of 4-5 people each. You may request one person to be in the same group as you; please submit your request in the survey (coming soon).
Canvas discussion. Please use the Canvas discussion forum liberally to post any questions or discussion.
Enrollment. Everyone who wants to will get in off the waitlist. Anyone not officially enrolled is welcome to view the lectures and assignments. If you would like to attend the interactive sessions, please respond to the survey (coming soon) and indicate that you would like to be assigned to a group with other auditors.
Grading criteria are broken down as follows: 65% written assignments; 25% coding assignments; 10% attendance. You get two free missed attendance days.
Overleaf. A great web-based Latex editor.
Collaboration policy: You may freely discuss the problem sets and coding assignments with other students. All writing must be your own; it is not acceptable to copy/paste or verbatim transcribe others' text, code or LaTeX source.
Other good resources: