Overview
Recent developments in neural networks (aka “deep learning”) have drastically advanced the performance of machine perception systems in a variety of areas including computer vision, robotics, and human shape modeling This course is a deep dive into deep learning algorithms and architectures with applications to a variety of perceptual and generative tasks.
Announcements
 04.05.2023
 The Machine Perception lectures on 10. / 11. May will be fully remote. Please join and ask your questions on Zoom. Exercise classes take place as usual.
 03.05.2023
 The mock exam is now available here.
 14.04.2023
 The Machine Perception lectures on 19. / 20. April will be fully remote. Please join and ask your questions on Zoom. Exercise classes take place as usual.
 28.03.2023
 The lectures this week are cancelled. Please use last years recordings (part I and part II) to study the material.
The tutorial will still take place on Thursday.There won't be tutorials this week.  23.02.2023
 Project descriptions have been added here!
 11.01.2023
 More info coming soon!
Learning Objectives
Students will learn about fundamental aspects of modern deep learning approaches for perception and generation. Students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cuttingedge research in learningbased computer vision, robotics, and shape modeling. The optional final project assignment will involve training a complex neural network architecture and applying it to a realworld dataset.
The core competency acquired through this course is a solid foundation in deeplearning algorithms to process and interpret humancentric signals. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multimodal data, among others.
We will focus on teaching: how to set up the problem of machine perception, the learning algorithms, network architectures, and advanced deep learning concepts in particular probabilistic deep learning models.
The course covers the following main areas:
I) Foundations of deep learning.
II) Advanced topics like probabilistic generative modeling of data (latent variable models, generative adversarial networks, autoregressive models, invertible neural networks).
III) Deep learning in computer vision, humancomputer interaction, and robotics.
Schedule
Subject to change. Materials only available from within ETH network.
Wk.  Date  Content  Material  Exercise Session 

1  22.02 
Deep Learning IntroductionClass content & admin 
slides 

1  23.02 
 No Class  

2  01.03 02.03 
Training Neural NetworksBackpropagation 
slides pt. I slides pt. II Perceptron Visualization Notebook 
Tutorial Implement your own MLP slidesXOR Notebook XOR Solutions EyeGaze Notebook EyeGaze Solutions Tutorial Linear Regr. slidesLinear Regression Notebook Pen & Paper Backprop. exerciseexercise solution 
3  09.03. 10.03. 
Convolutional Neural Networks 
Additional material:

Tutorial CNNs in Pytorch slidesCNN Notebook Pen & Paper CNN exerciseexercise solution 
4  15.03. 
Fully Convolutional Neural Networks 
slides  
4  16.03. 
Recurrent Neural NetworksLSTM, GRU, Backpropagation through time 
slides 
Tutorial RNNs in Pytorch slidesRNN Notebook Pen & Paper RNN exerciseexercise solution 
5  23.03. 24.03 
Generative Models Pt. I: Latent Variable ModelsVariational Autoencoders, etc. 
Class Tips for Training I slidesPen & Paper VAE exerciseexercise solution 

6  30.03. 31.03. 
Generative Models Pt. II: Autoregressive ModelsPixelCNN, PixelRNN, WaveNet, Stochastic RNNs 
slides pt. I slides pt. II 
Class Tips for Training II slidesPen & Paper AR exerciseexercise solution 
7  05.04. 06.04. 
Generative Models Pt. III: Normalizing Flows and Invertible Neural Networks

Pen & Paper NF exerciseexercise solution 

8  12.04. 23.04. 
 No Class (Easter)  

9  19.04. 20.04. 
Generative Models Pt. IV: Implicit ModelsGenerative Adversarial Networks & Co 
slides GAN Pt. I
slides GAN Pt. II 
Tutorial Exercise Discussion and Euler slides PP Backprop&CNNslides Euler Pen & Paper GAN exerciseexercise solution 
10  26.04 27.04 
Generative Models Pt. V: GAN Applications and Diffusion Models 
slides GAN Pt. III
slides Diffusion Models 
Pen & Paper class Diffusion Models exerciseexercise solution 
11  03.05 04.05 
Implicit Surfaces and Neural Radiance Fields 
slides NIR Pt. I
slides NIR Pt. II 
Tutorial Exercise Discussion AR slidesPen & Paper class Implicit Surfaces exerciseexercise solution 
12  10.05. 11.05. 
Parametric Human Body Models and Applications

slides PBM

Tutorial Exercise Discussion GAN & NF slidesPen & Paper PBM exerciseexercise solutions 
13  17.05. 18.05. 
 No classes or exercise sessions  

13  24.05. 25.05. 
Reinforcement Learning 
slides RL pt. I slides RL pt. II 
Pen & Paper RL exerciseexercise solutions 
14  01.06 
AIT Open House 

07.06 
Exam (13:30  16:30; HIL E 3, HIL E 4) 

27.07 / 28.07 
Exam Review (14:00  15:00; Location ML F 39) 
Exercise Sessions
Please refer to the above schedule once available for an overview of the planned exercise slots. We will have three different types of activities in the exercise sessions:
 Tutorial: Interactive programming tutorial in Python taught by a TA. Code will be made available.
 Class: Lecturestyle class taught by a TA to give you some tips on how to train your neural network in practice.
 Pen & Paper: Pen & paper exercises that are not graded but are helpful to prepare for the written exam. Solutions will be published on the website a week after the release and discussed in the exercise session if desired.
Project
Overview
There will be a multiweek project that gives you the opportunity to have some handson experience with training a neural network for a concrete application.
The project grade will be determined by two factors: 1) a competitive part based on how well your model fairs compared to your fellow students' models and 2) the idea/novelty/innovativeness of your approach based on a written report to be handed in by the project deadline. For each project there will be baselines available that guarantee a certain grade for the competitive part if you surpass them. The competition will be hosted on a online platform  more details will be announced here.
Check out the project descriptions here (you will need to log in with your ETH LDAP).
Exam
To give you a rough idea what to expect for the exam, we release a mock exam which you can download here:
Registration as Nonprimary Target Group
Registrations have been closed.