Spring Semester 2020

Simon Fraser University

Instructor: Oliver Schulte

Breadth Area III

Course Logistics

Office Location: TASC 1 9021.

Office Phone: 778-782-3390.

Office Hours: Wednesday 2pm-3 pm.

Office Hour: Monday 2-3 pm.

Email: myfirstname_mylastname@sfudotca

TA: Kiarash Zahirnia

E-mail: kzahirni@sfu_Email_Domain

TA Office Hour AND Location: The last Friday before each assignment due date, 5-6 pm; ASB 9808.

Inactive links are under construction

Announcements

Background quiz will take place in class next Monday January 13. Covers calculus and linear algebra.

Course Resources

• Schedule.
• Syllabus. Updated Dec 2, 2019.
• Textbook website.
• Installing Canvas Mobile. You need to install canvas to do the in-class quizzes.
• SFU Medical Excuse Form.

Slides

• Chapter 1: Linear classifiers
  • Gradient Descent Example Spreadsheet
• Neural Networks and Backpropagation
• More on Neural Net Training
  • Stanford Advice on Babysitting the Learning Process
• Convolutional Neural Networks
• Recurrent Neural Networks
  • Stanford Examples for Recurrent Neural Networks
• Embedding and Encoding
• Basic Auto-Encoder.
  • Zoom Lecture Recording 1. You may need the meeting password I sent earlier.
  • Zoom Lecture Recording 2. You may need the meeting password I sent earlier.
  • IPython Notebook for PCA demo
• Generative Models. Convolutional Auto-Encoder. Variational Auto-Encoder. Zoom Lecture Recording 3
• Transformers. State-of-the-art Sequence-to-Sequence Model. Zoom Lecture Recording 4
• Reinforcement Learning. Recording 1 Recording 2
• Deep Reinforcement Learning. Great topic but not on final exam.Recording

Homework Exercises

• Homework One
• Homework Two
• Homework Three. To go with Assignment 4.

Assignments

• Assignment One pdf version; and ipynb version
• Assignment Two html version; and ipynb version. Dataset
• Assignment Three html version; and ipynb version. 20Newsgroups and English Literature Datasets.
• Assignment Four html version; and ipynb version. Also hand in Homework 3 (one question only).

If you need additional computational resources (GPU, more memory), you can consider signing up for Google's collaborative computing environment.

Exam Information and Resources

• Final Exam Info: W April 15, 10:30 am – 12:30 pm, AQ 5037
• Midterm Information
• Midterm Instructions and Grading Breakdown
• Sample Midterm University of Toronto 2019
  • Solution to Backpropagation Question
• Sample Final Exam. University of Toronto 2018. You should try to solve this first yourself, you can check your solutions here.
• 30 questions to test a data scientist on Deep Learning
• Final Exam Info
  • Canvas Discussion Forum Screenshot
  • Zoom Meeting Recording. Final Exam discussion starts about 50 min into the meeting.
  • Final Exam Long Answer Questions
  • Final Exam Instructions

Learning Resources

Websites

• Deep Learning Website

• Deep Learning Software, including a Matlab Toolbox

• Stanford Course on Neural Nets for Vision.

Books

• Recent book on deep learning By Bengio, Goodfellow, and Courville. Covers many topics, a good reference to get a quick idea on what a deep learning approach to a machine learning problem would be. Digital edition.

• Pattern Recognition and Machine Learning, Chris Bishop, Springer

• Pattern Classification, Duda, Hart, and Stock, Wiley. See especially "Practical Considerations for Neural Net Learning".

• Nonlinear Dimensionality Reduction