Advanced Interactive Technologies, ETH Zürich

Prof. Dr. Otmar Hilliges

Professor of Computer Science
Advanced Interactive Technologies Lab, ETH Zürich

E-Mail: Please contact Monika Naegeli
Phone: +41 44 632 39 56
Address: Stampfenbachstrasse 48, 8092 Zürich, Switzerland
Room: ETH Zurich, Department of Computer Science, STD, H 24
Office Hour: Please register here for a 15-minutes slot.
Intended for disucssions about research or study with ETH students.

I am a Professor in the Computer Science Department at ETH Zurich. I lead the AIT lab and serve as the head of the Institute of Intelligent Interactive Systems.

My research is in spatio-temporal understanding of how humans move within and interact with the physical world. I research algorithms, methods and representations for human- and interaction-centric understanding of our world from videos, images and other sensor data. I'm interested in many different application domains such as Augmented and Virtual Reality, Human Robot Interaction and more.

Prior to joining ETH I was a postdoc and later a researcher at Microsoft Research Cambridge, in the I3D group(2010-2013). I was awarded a Diplom (equiv. MSc) in Computer Science from Technische Universität München and my PhD in Computer Science from LMU München, Germany (2009).

CV, Research and Teaching statements

Academic and Professional Activities

Grants

ERC Consolidator Grant: "AI-Perceive: Robust Human-Centric Computer Vision for Advanced AI-Agents". 2022-2027
Google Inc. Sponsored Research Agreement: "Generative Modelling of Humans". 2020-2025.
ERC Starting Grant: "OPTINT: Optimization-based Design of Interactive Technologies". 2017-2021
Microsoft Research Swiss JRC Grant: "Human-Centric-Flight II: End-user Design of High-level Robotic Behavior". 2017-2019
SNF Grant: "Deformation and Motion Modeling using Modular, Sensor-based Input Devices". 2015-2018
Google Inc. Sponsored Research Agreement: "Recognition Algorithms Using High-Speed, Wide Field-of-View, Short Range Sensors for Mobile and Wearable Computing". 2014-2016.
SNF Grant: "UFO: Semi-Autonomous Aerial Vehicles for Augmented Reality, Human-Computer Interaction and Remote Collaboration" (200021L_153644). 2015-2018
Microsoft Research Swiss JRC Grant: "Human-Centric Flight: Micro Aerial Vehicles for Interaction, Videography and 3D Reconstruction". 2014-2017

Technical Program Comittee Member

European Conference on Computer Vision (ECCV '22)
Conference on Computer Vision and Pattern Recognition (CVPR '23)
Conference on Computer Vision and Pattern Recognition (CVPR '22)
ACM SIGCHI Conference on Human Factors in Computing Systems (ACM CHI '20)
AAAI Conference on Artificial Intelligence (AAAI '20)
Eurographics (EG '20)
ACM Symposium on User Interface Software and Technology (ACM UIST '20)
ACM Symposium on User Interface Software and Technology (ACM UIST '19)
ACM SIGCHI Conference on Human Factors in Computing Systems (ACM CHI '17)
ACM Symposium on User Interface Software and Technology (ACM UIST '16)
ACM SIGCHI Conference on Human Factors in Computing Systems (ACM CHI '16)
ACM SIGCHI Conference on Human Factors in Computing Systems (ACM CHI '15)
ACM Symposium on User Interface Software and Technology (ACM UIST '14)
ACM SIGCHI Conference on Human Factors in Computing Systems (ACM CHI '14)
ACM Conference on Tangible, Embedded, and Embodied Interaction (ACM TEI '14)
ACM Symposium on User Interface Software and Technology (ACM UIST '13)
ACM SIGCHI Conference on Mobile and Ubiquitous Multimedia (ACM MUM '13)
ACM SIGCHI Conference on Human Factors in Computing Systems (ACM CHI '13)
ACM Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '13)
IEEE Symposium on Mixed and Augmented Reality (IEEE Ismar '13)
IEEE Joint 3DIM/3DPVT Conference (IEEE 3DV '13)
IEEE Joint 3DIM/3DPVT Conference (IEEE 3DIMPVT '12)
ACM Symposium on User Interface Software and Technology (ACM UIST '12)
ACM Conference on Interactive Tabletops and Surfaces (ACM ITS '10)
ACM Conference on Tangible and Embedded Interaction (ACM TEI '09)

Organizing Comittee Member

General Chair, ACM/IEEE Symposium on Mixed and Augmented Reality (ISMAR '18)
Keynote Chair, ACM Symposium on User Interface Software and Technology (ACM UIST '14)
Keynote Chair, ACM Symposium on User Interface Software and Technology (ACM UIST '13)
Video Co-Chair, ACM Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '13)
Demonstrations Co-Chair, ACM Symposium on User Interface Software and Technology (ACM UIST '10 & '11)
Student Volunteer Chair, ACM Conference on Interactive Tabletops and Surfaces (ACM ITS '08 & '09)

Keynote speaker

Keynote, International Conference on 3D Vision (3DV ' 22)
Keynote, Graphics Interface (GI ' 19)

Reviewing

I routinely review paper for conferences and journals in computer vision (CVPR, ICCV, 3DV, TPAMI), in HCI (CHI, UIST, Ubicomp, VR, ISMAR) and Robotics (ICRA, IROS, RA-L).

Selected Publications

Please see our full publications list. Also check Google Scholar, DBLP or the ACM DL.

PointAvatar: Deformable Point-based Head Avatars from Videos

AuthorsY. Zheng, W. Yifan, G. Wetzstein, M. Black, O. Hilliges
In ProceedingsarXiv, 2022

Prof. Dr. Otmar Hilliges

CV, Research and Teaching statements

Academic and Professional Activities

Grants

Technical Program Comittee Member

Organizing Comittee Member

Keynote speaker

Reviewing

Selected Publications

PointAvatar: Deformable Point-based Head Avatars from Videos

LiP-Flow: Learning Inference-time Priors for Codec Avatars via Normalizing Flows in Latent Space

EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

gDNA: Towards Generative Detailed Neural Avatars

I M Avatar: Implicit Morphable Head Avatars from Videos

PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence

VariTex: Variational Neural Face Textures

EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

Shape-aware Multi-Person Pose Estimation from Multi-view Images

PeCLR: Self-Supervised 3D Hand Pose Estimation from monocular RGB via Equivariant Contrastive Learning

SPEC: Seeing People in the wild with an Estimated Camera

PARE: Part Attention Regressor for 3D Human Body Estimation

CoSE: Compositional Stroke Embeddings

Self-Learning Transformations for Improving Gaze and Head Redirection

Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback

Capturing Subjective First-Person View Shots with Drones for Automated Cinematography

Human Body Model Fitting by Learned Gradient Descent

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints

ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation

Towards End-to-end Video-based Eye-Tracking

Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks

Monocular Neural Image Based Rendering with Continuous View Control

End-to-end Learning for Graph Decomposition

Few-Shot Adaptive Gaze Estimation

Structured Prediction Helps 3D Human Motion Modelling

Learning Cooperative Personalized Policies from Gaze Data

Context-Aware Online Adaptation of Mixed Reality Interfaces

Interactive Hand Pose Estimation using a Stretch-Sensing Soft Glove

Deformation Capture via Self-Sensing Capacitive Arrays

STCN:Stochastic Temporal Convolutional Networks

Real-time Environment-independent Multi-view Human Pose Estimation with Aerial Vehicles

Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time

Sample Efficient Learning of Path Following and Obstacle Avoidance Behavior for Quadrotors

DextrES: Wearable Haptic Feedback for Grasping in VR via a Thin Form-Factor Electrostatic Brake

Deep Pictorial Gaze Estimation

Optimizing for Aesthetically Pleasing Quadrotor Camera Motion

Cross-modal deep variational hand pose estimation

DeepWriting: Making Digital Ink Editable via Deep Generative Modeling

AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time

MagTics: Flexible and Thin Form Factor Magnetic Actuators for Dynamic and Wearable Haptic Feedback

Real-time Planning for Automated Multi-View Drone Cinematography

Thin-slicing network: A deep structured model for pose estimation in videos

Real-time Motion Planning for Aerial Videography with Dynamic Obstacle Avoidance and Viewpoint Optimization

Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum

Rig Animation with a Tangible and Modular Input Device

Airways: Optimization-based Planning of Quadrotor Trajectories according to High-Level User Goals

DefSense: Computational Design of Customized Deformable Input Devices

In-air Gestures Around Unmodified Mobile Devices

Environment-independent Formation Flight for Micro Aerial Vehicles

Tangible and Modular Input Device for Character Articulation

Type–Hover–Swipe in 96 Bytes: A Motion Sensing Mechanical Keyboard

Digits: Freehand 3D Interactions Anywhere Using a Wrist-worn Gloveless Sensor

HoloDesk: Direct 3D Interactions with a Situated See-Through Display