CMPT 727
CMPT 727: Statistical Machine Learning. Spring 2026.
Course logistics
- Tuesdays 10:00AM - 12:20PM, ASB 9896.
Instructor: Maxwell Libbrecht
- Web page
- Office: TASC 1 9219
- Email: maxwl at sfu dot ca
Recorded lectures: YouTube playlist. Videos will be added to this playlist in the order I intend for them to be watched. Lecture slides (may not match the videos exactly). Last year's lectures.
Discord discussion server: Invite link
Textbooks (please use the versions linked here in order for the sections/pages to match):
- PMLAI: "Probabilistic Machine Learning: An Introduction" by Kevin Patrick Murphy.
- PMLAT: "Probabilistic Machine Learning: Advanced Topics" by Kevin Patrick Murphy.
- Previous textbook, no longer used: MLPP: "Machine Learning: a Probabilistic Perspective" by Kevin Patrick Murphy.
Schedule
Week 1 (Jan 6):
- Introductory lecture in class.
- Quiz 1: qualifying quiz.
Week 2 (Jan 13):
- Reading: PMLAI Ch1 (p1-28), Ch2.1-6 (p33-61), LaTeX tutorial.
- Lectures: 1, 1.1, 5, 6.
- Week 2 quiz.
- Week 2 practice problems.
Week 3 (Jan 20):
- Reading: PMLAI Ch3.1-2 (p77-85), Ch3.6 (p99-102), A.4 (p774-776).
- Lectures: 7, 8, 9.1 .
- Week 3 quiz.
- Week 3 practice problems.
Week 4 (Jan 27):
- Reading: PMLAI Ch4.1-3 (p107-116), Ch4.5 (p120-127), Ch4.6.1-2 (p129-136), Ch4.6.7-8 (p147-154), Ch4.7.1 (p154). Ch5.1 (p167-178).
- Lectures: 9, 10, 2, 4, 4.1, 4.2, 18.
- Week 4 quiz.
- Week 4 problem set.
- Project proposal: Due in class today. 3-5 pages. The proposal should describe: (1) The goal of your project. (2) The data sets you will use. Include a table of all relevant statistics of the data set: number of examples, number of features per example, number of examples of each class, etc. Include at least one relevant plot of the data showing that the data takes the form you think it does (e.g. a histogram of feature values). (3) The ML/stats methods you will apply. You can expect this to change as you learn new methods in class, but please precisely describe at least one method and how you will apply it.
Week 5 (Feb 3):
- Reading: PMLAI 8.1-5, optimization; first-, second- and constrained methods.
- Lectures: 11, 12, 13, 14
- Week 5 quiz.
- Week 5 problem set.
Week 6 (Feb 10):
- Reading: PMLAI 8.7-8, bound optimization, EM and black-box.
- Lectures: 15.1, 15.2, 16, 17
- Programming assignment 1: Due Feb 10. Assignment. Starter code.
- Week 6 quiz.
- Week 6 problem set.
(Feb 17: off for Reading Break)
Week 7 (Feb 24):
- Reading: PMLAI 9.1-4 (LDA, Naive Bayes). 10.1-2 (logistic regression), 11.1-4 (linear regression, L2/L1 regularization).
- Lectures: 19,20,21,22,23,24,25,26,27,28.
- Week 7 quiz.
- Week 7 problem set.
Week 8 (Mar 3):
- Reading: PMLAI 11.6 (Robust regression). [Note the textbook switch.] PMLAT 4.1-2 (Bayesian networks).
- Lectures: 27.1, 28, 29, 29.1.
- Project milestone: Due in class today (Mar 3). 2-3 pages. In the milestone, you must present a working prototype of your approach, e.g., implement and evaluate one potential method. Describe this prototype and how you evaluated it, and present at least one plot of the results.
- Week 8 quiz.
- Week 8 problem set.
- Programming assignment 2: [EDIT] Due Mar 31. Assignment. Starter code.
Week 9 (Mar 10):
- Reading: PMLAT 4.3,4.5 (Markov random fields). PMLAT Ch7.1-3: Inference.
- Lectures: 30,31,32,33.
- Week 9 quiz.
- Week 9 problem set.
Week 10 (Mar 17):
- Reading: PMLAT Ch9.1-3 (Exact inference, belief propagation), 9.4 (Loopy BP), Ch9.5 (Variable elimination), 9.6 (Junction tree algorithm).
- Lectures: 33.1, 34, 34.1, 35, 35.1, 35.2 .
- Week 10 quiz.
- Week 10 problem set.
Week 11 (Mar 24):
- Reading: PMLAT 11.1-4 (Rejection sampling), 12.1-3 (Gibbs sampling, MCMC). 12.6 (Practical MCMC).
- Lectures: 36, 38, 39, 39.1, 40 .
- Week 11 quiz.
- Week 11 problem set.
Week 12 (Mar 31):
- Lectures: 41, 42.
- Reading: PMLAT 10.1.1-2, 10.3.1-2.
- Programming assignment 2: Due this week in class.
- Week 12 quiz.
- Week 12 problem set.
Week 13 (Apr 7): Last day of class.
- Final project report: Due Fri Apr 10, 11:59pm. 6-10 pages (not including references), in conference/journal format: Introduction/Background, Methods, Results, Discussion/Future Work. You may include an appendix beyond 10 pages, but I will focus my evaluation on the main content.
- Week 13 quiz.
Course information
Why we're offering this course: All machine learning methods have common underpinnings in probabilistic learning and inference. Probabilistic graphical models provide a unifying framework for expressing complex relationships among variables and for performing inference efficiently. By mastering this framework, students will gain the tools to represent and reason about complex systems. This perspective connects modern ML algorithms to their statistical foundations.
After this course you will be able to: (1) Design and implement probabilistic machine learning techniques including: probabilistic graphical models (Bayesian networks, Markov random fields), discrete and continuous distributions, EM algorithm learning, sampling-based inference methods (e.g. MCMC), variational inference, causal inference. (2) Understand and model dependence and causality among variables in complex systems. (3) Choose among machine learning and probabilistic modeling methods and anticipate their performance.
Who should take this course? This course is intended for graduate students or advanced undergraduates with an interest in machine learning or big data. You should take this course if: (1) You intend to use or develop machine learning in your research or work. (2) You are interested in fields where machine learning and big data is important including: vision, natural language processing/understanding, medical imaging, robotics, smart cities. (3) You want to understand machine learning at a deep level.
Unofficial prerequisites: No official prerequisites. However, the course assumes knowledge of machine learning (CMPT 726), probability (STAT 271) and linear algebra (MATH 251). The Assignment 1 is a qualifying assignment to show whether you have the necessary background.
How to enroll as an undergraduate: Please follow the following steps: (1) Come to the first lecture, where I will explain the expected background. (2) Email me stating that you have taken a course in probability/statistics and a course in machine learning (CMPT 726). If it is appropriate, I will respond with my approval. (3) Fill out the prerequisite waiver form and attach a screenshot of our email exchange.
Overview of course activities: This is a flipped-format course. You will do readings and watch recorded lectures at home. In class, we will do a quiz for the first hour, then work on assignments, which are practice problems for next week's quiz. There are two programming assignments and a final project. No midterm/final.
Quizzes. There will be a quiz each week at the start of class for the first hour. The quiz may contain questions from any topic up to this point. The quiz on a given week (e.g. Week 5) will contain somewhat easier questions on the current week's readings+lectures (Week 5 content), and somewhat harder questions focused on the previous week's readings+lectures (Week 4 content). Textbook, lectures and Wikipedia are allowed; no Google, AI tools or any other resources.
Assignments. In the remaining class time, you will work on problems in groups, which are intended as practice for next week's quiz. These are graded on completion: you get full points for making a reasonable effort on every problem.
Please format your quizzes and assignments using LaTeX (e.g. using Overleaf) and submit them on CourSys.
Grading breakdown: 50% quizzes. 15% practice problems. 15% programming assignments. 20% project (5% proposal, 5% milestone, 10% report). Grade cutoffs
Overleaf. A good free web-based Latex editor.
Late policy:
- Quizzes: No late submission. You get two free missed quizzes (your two lowest-score quizzes will be removed). If circumstances (e.g. illness) require you to miss more than two quizzes, please let me know.
- Project proposal and milestone: 75% grade for late submission.
- Practice problems, programming assignments, final project report: No late submission. Please email me at least 3 days in advance if you need an extension due to illness or other circumstances.
Collaboration policy: You may freely discuss the problem sets and coding assignments with other students. All writing must be your own; it is not acceptable to copy/paste or verbatim transcribe others' text, code or LaTeX source. You may consult AI tools for problem sets and coding assignments, but you may not use AI output verbatim.
Other good resources: