Overview
Recent developments in neural networks (aka “deep learning”) have drastically advanced the performance of machine perception systems in a variety of areas including computer vision, robotics, and human shape modeling This course is a deep dive into deep learning algorithms and architectures with applications to a variety of perceptual and generative tasks.
Announcements
- 03.05.2022
- There aren't any tutorials in May. If you have any questions about the lecture or the projects, please ask them on Moodle.
- 02.05.2022
- The mock exam is out. You can download it below.
- 26.04.2022
- This week's lectures (27.4/28.4) are canceled. There will be office hours for technical questions related to the projects in the tutorial slots.
- 22.03.2022
- This week's Thursday lecture has been moved to Friday. Please note: the lecture will take place in CAB G 11.The Friday tutorial will not take place. Instead, we stream the in-person session on Thursday via the usual zoom link.
- 16.03.2022
- Project registrations are open! Please find the link further below.
- 14.03.2022
- The class on 16.03 will be taught virtually by Jie Song. There will be a zoom live stream in the lecture hall (HG E 5).
- 09.03.2022
- Project registrations open next week. It's time to start looking for teammates!
- 02.02.2022
- There will be no class on 24.02. Tutorials take place.
- 03.01.2022
- More info coming soon!
Learning Objectives
Students will learn about fundamental aspects of modern deep learning approaches for perception and generation. Students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in learning-based computer vision, robotics, and shape modeling. The optional final project assignment will involve training a complex neural network architecture and applying it to a real-world dataset.
The core competency acquired through this course is a solid foundation in deep-learning algorithms to process and interpret human-centric signals. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multi-modal data, among others.
We will focus on teaching: how to set up the problem of machine perception, the learning algorithms, network architectures, and advanced deep learning concepts in particular probabilistic deep learning models
The course covers the following main areas:
I) Foundations of deep learning.
II) Advanced topics like probabilistic generative modeling of data (latent variable models, generative adversarial networks, auto-regressive models, invertible neural networks).
III) Deep learning in computer vision, human-computer interaction, and robotics.
Schedule
Subject to change. Materials only available from within ETH network.
| Wk. | Date | Content | Material | Exercise Session |
|---|---|---|---|---|
| 1 | 23.02 |
Deep Learning IntroductionClass content & admin |
slides |
Tutorial Implement your own MLP slidesXOR Notebook XOR Solutions Eye-Gaze Notebook Eye-Gaze Solutions |
| 1 | 24.02 |
-- No Class -- |
||
| 2 | 02.03 03.03 |
Training Neural NetworksBackpropagation |
slides pt. I slides pt. I (annotated) slides pt. II slides pt. II (annotated) Perceptron Visualization Notebook |
Tutorial Linear Regr. slidesLinear Regression Notebook Pen & Paper Backprop. exerciseexercise solution |
| 3 | 09.03. 10.03. |
Convolutional Neural Networks |
slides pt. I Additional material:
|
Tutorial CNNs in Pytorch slidesCNN Notebook Pen & Paper CNN exerciseexercise solution |
| 4 | 16.03. |
Fully Convolutional Neural Networks |
slides | |
| 4 | 17.03. |
Recurrent Neural NetworksLSTM, GRU, Backpropagation through time |
slides slides (annotated) |
Tutorial RNNs in Pytorch slidesRNN Notebook Pen & Paper RNN exerciseexercise solution |
| 5 | 23.03. 24.03 |
Generative Models Pt. I: Latent Variable ModelsVariational Autoencoders, etc. |
Class Tips for Training I slidesPen & Paper VAE exercise (2021)exercise (2021) solution exercise exercise solution |
|
| 6 | 30.03. 31.03. |
Generative Models Pt. II: Autoregressive ModelsPixelCNN, PixelRNN, WaveNet, Stochastic RNNs |
slides pt. I slides pt. I (annotated) slides pt. II |
Class Tips for Training II slidesPen & Paper AR exerciseexercise solution |
| 7 | 06.04. 07.04. |
Generative Models Pt. III: Normalizing Flows and Invertible Neural Networks
|
slides NF Pt. I
Additional material:
|
Pen & Paper NF exerciseexercise solution |
| 8 | 13.04. 14.04. |
Generative Models Pt. IV: Implicit ModelsGenerative Adversarial Networks & Co |
slides GAN Pt. I
slides GAN Pt. I (annotated) slides GAN Pt. II slides GAN Pt. II (annotated) |
Pen & Paper GAN exerciseexercise solution |
| 9 | 20.04. 21.04. |
-- No Class (Easter) -- |
||
| 10 | 27.04 28.04. |
Graph Neural Networks(canceled)
|
||
| 11 | 04.05. 05.05. |
Parametric Human Body Models and Applications
|
slides PBM pt. I
slides PBM pt. II |
Pen & Paper PBM exerciseexercise solutions |
| 12 | 11.05. 12.05. |
Implicit Surfaces and Neural Radiance Fields
|
slides NIR pt. I
slides NIR pt. II slides NIR pt. III |
Pen & Paper NIR exerciseexercise solutions |
| 13 | 18.05. 19.05. |
Reinforcement Learning |
slides RL pt. I slides RL pt. I (annotated) slides RL pt. II slides RL pt. II (annotated) |
Pen & Paper RL exerciseexercise solutions |
| 14 | 25.05. |
Recent ResearchOpen House Event @ Heinrichstrasse 237. Sign up for a tour slot here |
||
| 14 | 26.05. |
-- No Class (Ascension Day) -- |
||
| 15 | 01.06. |
-- No ClassPlease submit questions to Moodle. TAs will post answers on Moodle for all questions posted before 11:59am. |
||
| 15 | 02.06. |
Exam (12:00 - 15:00)HG F3 and HG G5 |
||
| - | 16.06. |
Project DeadlineMidnight local time. |
Exercise Sessions
Please refer to the above schedule once available for an overview of the planned exercise slots. We will have three different types of activities in the exercise sessions:
- Tutorial: Interactive programming tutorial in Python taught by a TA. Code will be made available.
- Class: Lecture-style class taught by a TA to give you some tips on how to train your neural network in practice.
- Pen & Paper: Pen & paper exercises that are not graded but are helpful to prepare for the written exam. Solutions will be published on the website a week after the release and discussed in the exercise session if desired.
The tutorials and classes cover the same content on Thursday and Friday. You can decide which day you want to attend. The Thursday session is in person (CAG G 11) and has 190 seats. The Friday session is on zoom and has no limit on the number of students.
Please find the zoom link for the exercises sessions on slide 39 of the first week's slide deck.
Project
Overview
There will be a multi-week project that gives you the opportunity to have some hands-on experience with training a neural network for a concrete application.
Check out the project descriptions here.
Project registrations are open! Please sign your group up here. Registrations will close on March 23rd, 11:59pm, please make sure to sign up in time.
Exam
To give you a rough idea what to expect for the exam, we release a mock exam which you can download here: