Overview
Recent developments in neural networks have drastically advanced the performance of machine perception systems in a variety of areas including computer vision, robotics, and human shape modeling. This course is a deep dive into deep learning algorithms and architectures with applications to a variety of perceptual and generative tasks.
Announcements
- 17.01.2025
- Website live - More info coming soon!
Learning Objectives
Students will learn about fundamental aspects of modern deep learning approaches for perception and generation. Students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in learning-based computer vision, robotics, and shape modeling. The optional final project assignment will involve training a complex neural network architecture and applying it to a real-world dataset.
The core competency acquired through this course is a solid foundation in deep-learning algorithms to process and interpret human-centric signals. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multi-modal data, among others.
The courses focuses on teaching how to set up the problem of machine perception and the associated learning algorithms, neural network architectures, and advanced deep learning concepts.
The course covers the following main areas:
I) Foundations of Deep Learning: Multilayer perceptrons, backpropagation, time-series modeling, convolutional neural networks.
II) Advanced topics: latent variable models, generative adversarial networks, auto-regressive models, invertible neural networks, normalizing flows, diffusion models, neural implicit surface representations, neural radiance fields.
III) Applications in machine perception and human-centric computer vision: general understanding of human activities, 3D reconstruction of human performance using different input modalities (monocular or multi-view images, body-worn sensors) and representations (explicit triangulated meshes, parametric body models, implicit surfaces, neural radiance fields, 3D Gaussian Splatting-based).
Lecture Notes
You can download the lecture notes here (you will need to log in with your ETH LDAP).
These lecture notes are provided as a draft version for educational purposes only. The content presented herein is subject to change and may contain inaccuracies or errors. Grading for the course will be based on slides and the lecture materials.
Schedule
Subject to change. Materials only available from within ETH network.
| Wk. | Date | Content | Material | Exercise Session |
|---|---|---|---|---|
| 1 | 19.02 |
Deep Learning IntroductionClass content & admin |
||
| 1 | 20.02 |
-- No Class -- |
Exercise Sessions
Please refer to the above schedule once available for an overview of the planned exercise slots. We will have three different types of activities in the exercise sessions:
- Tutorial: Interactive programming tutorial in Python taught by a TA. Code will be made available.
- Class: Lecture-style class taught by a TA to give you some tips on how to train your neural network in practice.
- Pen & Paper: Pen & paper exercises that are not graded but are helpful to prepare for the written exam. Solutions will be published on the website a week after the release and discussed in the exercise session if desired.
Project
There will be a multi-week project that gives you the opportunity to have some hands-on experience with training a neural network for a concrete application. The project is optional and to be completed within groups of 3-4 students. You will be able to select from three projects. Passing a pre-determined baseline will award you with a 0.25 grade bonus. There will be a leaderboard giving you the opportunity to compete against other groups. However, your rank in the leaderboard has no influence on the grade bonus (apart from passing the baseline). Project descriptions and more information will follow during the semester.
Exam
To give you a rough idea what to expect from the exam, we will release a mock exam here in due time.