# CMPT 727

**CMPT 727: Statistical Machine Learning. Spring 2021.**

# Course logistics

- Synchronous Zoom schedule: Thursdays 10:30AM-12:00PM (Vancouver time).
- (Officially scheduled: Mondays 10:30AM - 11:20AM, Thursdays 10:30AM - 12:20PM)
- Taught remotely.
- Video lectures: Youtube playlist. New videos will be added to this playlist.
- Dropbox with recordings and slides. Due to bandwidth limits, please download videos from Dropbox only if you cannot access Youtube.

# Zoom information

- Link: https://sfu.zoom.us/j/67258683175?pwd=RGp2YVI5U2NUZlVINDhUODg1Y01Ydz09
- Meeting ID: 672 5868 3175. Password: 245514.

Instructor: Maxwell Libbrecht

- Web page
- Office: TASC 1 9219
- Email: maxwl at sfu dot ca

TA: Heng Liu

- Email: liuhengl at sfu dot ca

Textbooks:

- PMLAI: "Probabilistic Machine Learning: An Introduction" by Kevin Patrick Murphy.
- MLPP: "Machine Learning: a Probabilistic Perspective" by Kevin Patrick Murphy.

# Programming Assignment

Code and data *Updated on Apr 13

Due April 26.

# Groups

Assignment Groups. Last Update Jan 27.

# Schedule

Finals period (Apr 17-28). No lectures. Final coding assignment, released Apr 5, due Apr 26 (subject to change due to grading deadlines). Python tutorial will be released Apr 5.

Week 13 (Apr 12). Last week of lectures. No written assignment this week.

- (No reading this week)
- Lectures: (coming soon)
- (No written assignment this week)

Week 12 (Apr 5). No written assignment this week.

- Reading MLPP 23.1-3 (Rejection sampling). MLPP 24.1-4 (Gibbs sampling and MCMC).
- Lectures: (coming soon)
- In-class exercise (nothing needs to be turned in):

Week 11 (Mar 29).

- Reading. MLPP 20.1-2 (Belief propagation). 20.3 (Variable elimination). 20.4.1 (Idea behind junction tree algorithm.) 20.5 (Complexity of inference.)
- Lectures: Recorded lectures 33-35.
- Assignment 11 Due Monday, Apr 5, 11:59pm. Latex source file.

Week 10 (Mar 22).

- Reading.
- Note: We are switching this week from PMLAI to MLPP. Skip MLPP sections marked with a "*"; these are optional sections with more complex derivations.
- Review MLPP Ch10.1-4 (Probabilistic graphical models). MLPP Ch10.5 (d-separation). MLPP Ch19.1-4 Undirected graphical models.

- Lectures: Recorded lectures 29-32.
- Assignment 10 Due Monday, Mar 29, 11:59pm. Latex source file.

Week 9 (Mar 15)

- Reading: Ch11.1-3 (linear regression, ridge regression). Ch11.5 (LASSO).
- Lectures: Recorded lectures 24-28.
- Assignment 9 Due Monday, Mar 22, 11:59pm. Latex source file.

Week 8 (Mar 8)

- Reading: Ch9.1-9.2.5 (LDA). 9.3-4 (Naive Bayes). Ch10.1-2, 4-5 (Logistic regression).
- Lectures: Recorded lectures 19-23.
- Assignment 8 Due Monday, Mar 15, 11:59pm. Latex source file.

Week 7 (Mar 1)

- Reading: 5.7 (Bound optimization and EM), 5.8 (Black box optimization), 8.1 (Bayesian decision theory).
- Lectures: Recorded lectures 15-18.
- Assignment 7 Due Monday, Mar 8, 11:59pm. Latex source file.

Week 6 (Feb 22)

- Reading: 5.1-2 (First-order optimization). 5.4-5.5.2 (SGD, constrained optimization).
- Lectures: Recorded lectures 11-14.
- Assignment 6 Due Monday, Mar 1, 11:59pm. Latex source file.

(Feb 15: Reading week.)

Week 5 (Feb 8)

- Reading: PMLAI Ch 4 (Parameter estimation).
- Lectures: Recorded lectures 9-10.
- Assignment 5 Due Monday, Feb 22, 11:59pm. Latex source file.

Week 4 (Feb 1)

- Reading: PMLAI Ch3.5-9 (Multivariate and linear Gaussian, mixture models, probabilistic graphical models).
- Lectures: Recorded lectures 7-8.
- Assignment 4 Due Monday, Feb 8, 11:59pm. Latex source file.

Week 3 (Jan 25)

- Reading: PMLAI Ch2.4 (Bayesian ML). PMLAI Ch3.1-3 (Univariate probabilisitic models).
- Lectures: Recorded lectures 4-6.
- Assignment 3 Due Monday, Feb 1, 11:59pm. Latex source file.

Week 2 (Jan 18)

- Reading: Learn LaTeX in 30 minutes. PMLAI Ch1. PMLAI Ch2.1-3.
- Lectures: LaTex Tutorial for Beginners (90 min Youtube Video). Recorded lectures 1, 1.1, 2, 3. (See Recorded Lectures link at the top of the page.)
- Special Office Hour for Latex questions: 10:30AM - 11:30AM Monday, Jan 18, Vancouver time. Zoom Link
- Assignment 2 Due Monday, Jan 25, 11:59pm. Latex source file. Must be formatted with Latex.

Week 1 (Jan 11)

- Monday lecture: Zoom; Meeting ID: 672 5868 3175. Password: 245514. Recording. Slides.
- Group assignment survey. Due Wed Jan 13 at 12pm Vancouver time.
- Assignment 1. Due Monday, Jan 18, 11:59pm (time zone of your choice). This is a qualifying assignment; based on your performance on the assignment, we will recommend whether you have the prerequisites for the course. (For the purposes of the course grade, everyone will get full credit, so please represent your knowledge accurately.) You should expect to consult reference materials (e.g. textbooks, Wikipedia, etc), but if you find you need to re-read whole textbook chapters, that is a sign that you are missing the prerequisite knowledge.

# Course information

Textbook: We will use a combination of two textbooks from Kevin Murphy.

- "Probabilistic Machine Learning: An Introduction" (2021). Available free online. We will use this for the first half of the course.
- "Machine Learning: A Probabilistic Perspective" (2012). We will use this for the section on probabilistic graphical models, the last half of the course.

- Reading assignments will be posted on this page, due before the interactive session on Thursday.

Lectures (subject to change):

- Content lectures will be recorded and posted, to be viewed asynchronously, due before the interactive session on Thursday.
- We will have one synchronous interactive session per week, Thursdays 10:30am-Noon Vancouver time.
- Attendance is mandatory at the interactive session. If this is challenging for you, e.g. due to time zone differences, please indicate in the survey (coming soon).
- Depending on feedback, synchronous lecture times are subject to change after the first few weeks. Please reserve the full scheduled class times.

Assignments. There will be one written assignment per week, due on Monday.

- Assignments will be due in two parts. Your first submission is individual, worth 2/3 of the grade.
- The second submission, due one week later, is a joint group submission, worth 1/3 of the grade. This gives you a chance to fix any mistakes as a group. (Group grade is the maximum of the individual and group submission.)

Groups. The class will be divided into groups of 4-5 people each. You may request one person to be in the same group as you; please submit your request in the survey (coming soon).

Canvas discussion. Please use the Canvas discussion forum liberally to post any questions or discussion.

Enrollment. Everyone who wants to will get in off the waitlist. Anyone not officially enrolled is welcome to view the lectures and assignments. If you would like to attend the interactive sessions, please respond to the survey (coming soon) and indicate that you would like to be assigned to a group with other auditors.

Grading criteria are broken down as follows: 75% written assignments (11 assignments, 6.8% each); 15% final coding assignment; 10% attendance. You get two free missed attendance days.

Unofficial prerequisites: No official prerequisites. However, the course assumes knowledge of machine learning (CMPT 726), probability and linear algebra (MATH 240). The Assignment 1 is a qualifying assignment to show whether you have the necessary background.

The course is open to advanced undergraduates with permission. How to enroll as an undergraduate: Please follow the following steps: (1) Come to the first lecture, where I will explain the expected background. (2) Email me stating that you have taken a course in probability/statistics and a course in machine learning (CMPT 726). If it is appropriate, I will respond with my approval. (3) Fill out the prerequisite waiver form and attach a screenshot of our email exchange.

Overleaf. A great web-based Latex editor.

Other good resources.