Dr. Manuel Kaufmann

Postdoc
manuel.kaufmann@inf.ethz.ch
ETH Zurich, Department of Computer Science, STD, H 29.1
Stampfenbachstrasse 48, 8092 Zürich, Switzerland

About me

I am a tech lead at the ETH AI Center and a Postdoc at the AIT Lab. I joined the Advanced Interactive Technologies lab in December 2017 as a Ph.D. student under the supervision of Prof. Otmar Hilliges. I received my BSc in Computer Science from the University of Basel (2014) where I wrote my Bachelor thesis under the supervision of Prof. Thomas Vetter on "A Statistical Shape Model of the Human Ear to Customize Hearing Devices". I subsequently obtained a MSc in Computer Science at ETH Zürich (2017). My Master thesis was titled "A Deep Learning Approach to Human Motion Sequences Infilling" supervised by Prof. Hilliges and in collaboration with vizrt Switzerland.

I am interested in human-centric data acquisition pipelines, 3D human pose estimation from wearable sensors such as IMUs and electromagnetic sensors, 3D reconstruction of human avatars from images and human motion modelling, for example to predict motions into the future, fill in missing frames or clean up noisy data.

You can also find me on Twitter, Linkedin, and Google scholar.

Publications

EgoHDM: An Online Egocentric-Inertial Human Motion Capture, Localization, and Dense Mapping System
Bonan Liu*, Handi Yin*, Manuel Kaufmann, Jinhao He, Sammy Christen, Jie Song, Pan Hui * Equal contribution
SIGGRAPH Asia 2024 (TOG), 2024
Video Paper
ReLoo: Reconstructing Humans Dressed in Loose Garments from Monocular Video in the Wild
Chen Guo*, Tianjian Jiang*, Manuel Kaufmann, Chengwei Zheng, Julien Valentin, Jie Song, Otmar Hilliges * Equal contribution
European Conference on Computer Vision (ECCV), 2024
Video Paper
WorldPose: A World Cup Dataset for Global 3D Human Pose Estimation
Tianjian Jiang, Johsan Billingham, Sebastian Müksch, Juan Zarate, Nicolas Evans, Martin Oswald, Marc Pollefeys, Otmar Hilliges, Manuel Kaufmann, Jie Song
European Conference on Computer Vision (ECCV), 2024
Paper
HSR: Holistic 3D Human-Scene Reconstruction from Monocular Videos
Lixin Xue, Chen Guo, Chengwei Zheng, Fangjinhua Wang, Tianjian Jiang, Hsuan-I Ho, Manuel Kaufmann, Jie Song, Otmar Hilliges
European Conference on Computer Vision (ECCV), 2024
Video Paper
MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild
Zeren Jiang*, Chen Guo*, Manuel Kaufmann, Tianjian Jiang, Julien Valentin, Otmar Hilliges, Jie Song * Equal contribution
Computer Vision and Pattern Recognition (CVPR), 2024
Accepted as Oral Presentation (Top 3.3%)
Video Paper Code
EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild
Manuel Kaufmann, Jie Song, Chen Guo, Kaiyue Shen, Tianjian Jiang, Chengcheng Tang, Juan Zarate, Otmar Hilliges
International Conference on Computer Vision (ICCV), 2023
Video Paper Data Code
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael Black, Otmar Hilliges
Computer Vision and Pattern Recognition (CVPR), 2023
Video Paper Code
X-Avatar: Expressive Human Avatars
Kaiyue Shen*, Chen Guo*, Manuel Kaufmann, Juan Zarate, Julien Valentin, Jie Song, Otmar Hilliges * Equal contribution
Computer Vision and Pattern Recognition (CVPR), 2023
Video Paper Data Code
Hi4D: 4D Instance Segmentation of Close Human Interaction
Yifei Yin, Chen Guo, Manuel Kaufmann, Juan Zarate, Jie Song, Otmar Hilliges
Computer Vision and Pattern Recognition (CVPR), 2023
Video Paper Data Code
A Spatio-temporal Transformer for 3D Human Motion Prediction
Emre Aksan, Manuel Kaufmann, Peng Cao, Otmar Hilliges
International Conference on 3D Vision (3DV), 2021
Video Paper Code
EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers
Manuel Kaufmann, Yi Zhao, Chengcheng Tang, Lingling Tao, Christopher Twigg, Jie Song, Robert Wang, Otmar Hilliges
International Conference on Computer Vision (ICCV), 2021
Video Paper Data Code
Convolutional Autoencoders for Human Motion Infilling
Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, Otmar Hilliges
2020 International Conference on 3D Vision (3DV), 2020
Video Paper Code
Structured Prediction Helps 3D Human Motion Modelling
Emre Aksan*, Manuel Kaufmann*, Otmar Hilliges * Equal contribution
The IEEE International Conference on Computer Vision (ICCV), 2019
Video Paper Code
Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time
Yinghao Huang*, Manuel Kaufmann*, Emre Aksan, Michael Black, Otmar Hilliges, Gerard Pons-Moll * Equal contribution
ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), 2018
Video Paper Data Code

Academic Activities

Reviewing

2024
CVPR, ECCV, SIGGRAPH, SIGGRAPH Asia
2023
CVPR, ICCV, 3DV
2022
CVPR, ECCV (outstanding reviewer), 3DV, SIGGRAPH Asia, ACM UIST
2021
ICCV, SIGGRAPH, SIGGRAPH Asia
2020
ACM UIST, CVPR, ECCV, Eurographics, WACV
2019
Biocybernetics and Biomedical Engineering

Student Theses Selection

2024
Zeren Jiang Semester Thesis
MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild
2023
Alexander Häfliger Master Thesis
Deep Kinematics: Uncovering Human Biomechanics with IMUs and Deep Learning
2022
Nina Weng Master Thesis
Explainability in Deep Learning for Prediction of Gaze Position from EEG Data
Yelan Tao Semester Project
Synthesis of Egocentric Views Showing Hand-Object Interactions
2021
Seok Joon Kim Master Thesis
Conditional Hand Motion Synthesis
2020
Dominik Bornand Semester Project
Simulating Sparse Inertial Measurements for Human Pose Reconstruction
2019
Jonathan Lehner Bachelor Thesis
Real-Time Hand Tracking from Wrist-Worn RGB Cameras
2018
Martin Blapp Bachelor Thesis
Evaluation of Human Motion Models

Teaching