Advanced Topics in Human-Centric Computer Vision

The learning objective is to analyze selected research papers published at top computer vision and machine learning venues. A key focus will be placed on identifying and discussing open problems and novel solutions in this space. The seminar will achieve this via several components: reading papers, technical presentations, writing analysis and critique summaries, class discussions, and exploration of potential research topics. In this seminar we will discuss state-of-the-art literature on human-centric computer vision topics including but not limited to human pose estimation, hand and eye-gaze estimation as well as generative modeliing of detailed human activities.


Overview

eDoz Course Nr.
263-3713-00L
Lecturer
Otmar Hilliges
Assistants
Jie Song, Manuel Kaufmann, Xu Chen, Sammy Christen, Gengyan Li, Marcel C. Bühler, Zicong Alex Fan, Yufeng Zheng, Muhammed Kocabas, Zijian Dong
Seminar
Thu 4-6pm, Virtual
ECTS credits
2 ECTS
Lecture Zoom
Zoom
Papers
Papers
Paper signup form
Signup
Slack
Slack
Review Template
Review Template



Goal

The goal of the seminar is not only to familiarize students with exciting new research topics, but also to teach basic scientific writing and oral presentation skills. The seminar will have a different structure from regular seminars to encourage more discussion and a deeper learning experience.

We will treat papers as case studies and discuss them in-depth in the seminar. Once per semester, every student will have to take one of the following roles:

  1. Presenter: Give a presentation about the paper that you read in depth.
  2. Reviewer: Write a critical review of the paper following this template.

All other students read the paper and submit questions they have about the paper before the presentation.




Topics


Topics will cover the following areas:

Human Pose Estimation

We will discuss papers about 3D human pose estimation from single- or multi-view images, IMUs or other sensor inputs.

Example papers:
4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras
Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors

Human-Object Interaction

We will discuss papers with topics such as inferring hand-object interation from images and synthesizing hand-object interation.

Example papers:
ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation
Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild

3D Human Modeling

This topic includes modelling the shape and texture of human bodies, faces and clothing.

Example papers:
Learning an Animatable Detailed 3D Face Model from In-The-Wild Images
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiesg
SMPLicit: Topology-aware Generative Model for Clothed People

Neural Rendering and Representation

This topic includes the application neural rendering techniques and neural representations of 3D scenes and objects in human related tasks.

Example papers:
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields
Neural Volumes: Learning Dynamic Renderable Volumes from Images

The full paper list can be found here. Please register your preference in this form


Schedule

Wk. Date TA Paper
1 23.09.2021
--
Introduction
Recording
2 30.09.2021
--
no seminar
3 07.10.2021
--
no seminar
4 14.10.2021
Jie Song
Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies
5 21.10.2021
Manuel Kaufmann
TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors
6 28.10.2021
Marcel Bühler
StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation
7 04.11.2021
Gengyan Li
The Eyes Have It: An Integrated Eye and Face Model for Photorealistic Facial Animation
8 11.11.2021
--
no seminar
9 18.11.2021
--
no seminar
10 25.11.2021
Sammy Christen
Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation
11 02.12.2021
Yufeng Zheng
ManipNet: Neural Manipulation Synthesis with a Hand-Object Spatial Representation
12 09.12.2021
Zicong Alex Fan
Reconstructing Hand-Object Interactions in the Wild
13 16.12.2021
Xu Chen
Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild
14 23.12.2019
--
no seminar