Prof. Dr. Otmar Hilliges

Prof. Dr. Otmar Hilliges

Professor of Computer Science
Advanced Interactive Technologies Lab, ETH Zürich

E-Mail
Please contact Monika Naegeli
Phone
+41 44 632 39 56
Address
Stampfenbachstrasse 48, 8092 Zürich, Switzerland
Room
ETH Zurich, Department of Computer Science, STD, H 24
Office Hour
Please register here for a 15-minutes slot.
Intended for disucssions about research or study with ETH students.


I am a Professor in the Computer Science Department at ETH Zurich. I lead the AIT lab and serve as the head of the Institute of Intelligent Interactive Systems.

My research is in spatio-temporal understanding of how humans move within and interact with the physical world. I research algorithms, methods and representations for human- and interaction-centric understanding of our world from videos, images and other sensor data. I'm interested in many different application domains such as Augmented and Virtual Reality, Human Robot Interaction and more.

Prior to joining ETH I was a postdoc and later a researcher at Microsoft Research Cambridge, in the I3D group(2010-2013). I was awarded a Diplom (equiv. MSc) in Computer Science from Technische Universität München and my PhD in Computer Science from LMU München, Germany (2009).




CV, Research and Teaching statements

Academic and Professional Activities

Grants

Technical Program Comittee Member

Organizing Comittee Member

Keynote speaker

Reviewing

I routinely review paper for conferences and journals in computer vision (CVPR, ICCV, 3DV, TPAMI), in HCI (CHI, UIST, Ubicomp, VR, ISMAR) and Robotics (ICRA, IROS, RA-L).



Selected Publications

Please see our full publications list. Also check Google Scholar, DBLP or the ACM DL.

Teaser-Picture

EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes

AuthorsG. Li, A. Meka, F. Mueller, M. Buehler, O. Hilliges, T. Beeler
In ProceedingsACM Trans. Graph. (Volume: 41, Issue: 4), Jul 2022
ProjectPDFVideoVideoBibTeX
Teaser-Picture

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

AuthorsS. Christen, M. Kocabas, E. Aksan, J. Hwangbo, J. Song, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022
ProjectCodePDFVideoBibTeX
Teaser-Picture

gDNA: Towards Generative Detailed Neural Avatars

AuthorsX. Chen, T. Jiang, J. Song, J. Yang, M. Black, A. Geiger, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022
ProjectCodePDFVideoBibTeX
Teaser-Picture

I M Avatar: Implicit Morphable Head Avatars from Videos

AuthorsY. Zheng, V. Abrevaya, M. Bühler, X. Chen, M. Black, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022

Accepted as Oral Presentation
ProjectCodePDFVideoBibTeX
Teaser-Picture

PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence

AuthorsZ. Dong*, C. Guo*, J. Song, X. Chen, A. Geiger, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX

VariTex: Variational Neural Face Textures

AuthorsM. C. Buehler, A. Meka, G. Li, T. Beeler, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

AuthorsM. Kaufmann, Y. Zhao, C. Tang, L. Tao, C. Twigg, J. Song, R. Wang, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

AuthorsX. Chen, Y. Zheng, M. Black, O. Hilliges, A. Geiger
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

Shape-aware Multi-Person Pose Estimation from Multi-view Images

AuthorsZ. Dong, J. Song, X. Chen, C. Guo, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

PeCLR: Self-Supervised 3D Hand Pose Estimation from monocular RGB via Equivariant Contrastive Learning

AuthorsA. Spurr*, A. Dahiya*, X. Wang, X. Zhang, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
* These two authors contributed equally to this work.

Accepted as Oral Presentation
ProjectCodePDFVideoBibTeX
Teaser-Picture

SPEC: Seeing People in the wild with an Estimated Camera

AuthorsM. Kocabas, C. Huang, J. Tesch, L. Müller, O. Hilliges, M. Black
In ProceedingsProceedings of the International Conference on Computer Vision 2021, Oct 2021
ProjectBibTeX
Teaser-Picture

PARE: Part Attention Regressor for 3D Human Body Estimation

AuthorsM. Kocabas, C. Huang, O. Hilliges, M. Black
In ProceedingsProceedings of the International Conference on Computer Vision 2021, Oct 2021
ProjectPDFBibTeX

CoSE: Compositional Stroke Embeddings

AuthorsE. Aksan, T. Deselaers, A. Tagliasacchi, O. Hilliges
In ProceedingsAdvances in Neural Information Processing Systems, 2020
ProjectCodePDFBibTeX

Self-Learning Transformations for Improving Gaze and Head Redirection

AuthorsY. Zheng, S. Park, X. Zhang, S. De Mello, O. Hilliges
In ProceedingsNeural Information Processing Systems (NeurIPS), 2020
ProjectCodePDFBibTeX
Teaser-Picture

Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback

AuthorsT. Langerak*, J. Zarate*, D. Lindlbauer, C. Holz, O. Hilliges
In ProceedingsProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, Virtual Event, USA, 2020
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Capturing Subjective First-Person View Shots with Drones for Automated Cinematography

AuthorsA. Ashtari, S. Stevsic, T. Nägeli, J. Bazin, O. Hilliges
In ProceedingsACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 39, Issue: 5), Aug 2020
ProjectPDFVideoBibTeX
Teaser-Picture

Human Body Model Fitting by Learned Gradient Descent

AuthorsJ. Song*, X. Chen*, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

AuthorsX. Chen*, Z. Dong*, J. Song, A. Geiger, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints

AuthorsA. Spurr, U. Iqbal, P. Molchanov, O. Hilliges, J. Kautz
In ProceedingsEuropean Conference on Computer Vision (ECCV), Glasgow, Scotland, 2020

Winner of HANDS2019 challenge: RGB-based 3D Hand Pose Estimation while Interacting with Objects
ProjectBibTeX
Teaser-Picture

ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation

AuthorsX. Zhang, S. Park, T. Beeler, D. Bradley, S. Tang, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020

Accepted as Spotlight Presentation
ProjectCodePDFBibTeX

Towards End-to-end Video-based Eye-Tracking

AuthorsS. Park, E. Aksan, X. Zhang, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
ProjectCodePDFBibTeX
Teaser-Picture

Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks

AuthorsZ. He, A. Spurr, X. Zhang, O. Hilliges
In ProceedingsIEEE International Conference on Computer Vision (ICCV), Seoul, Korea, 2019
ProjectCodePDFBibTeX
Teaser-Picture

Monocular Neural Image Based Rendering with Continuous View Control

AuthorsX. Chen*, J. Song*, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019
* These two authors contributed equally to this work.
ProjectCodePDFBibTeX
Teaser-Picture

End-to-end Learning for Graph Decomposition

AuthorsJ. Song, B. Andres, M. Black, O. Hilliges, S. Tang
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019
ProjectPDFBibTeX

Few-Shot Adaptive Gaze Estimation

AuthorsS. Park*, S. De Mello*, P. Molchanov, U. Iqbal, O. Hilliges, J. Kautz
In ProceedingsInternational Conference on Computer Vision (ICCV), Seoul, Korea, 2019
* These two authors contributed equally to this work.

Accepted as Oral Presentation
ProjectCodePDFVideoBibTeX
Teaser-Picture

Structured Prediction Helps 3D Human Motion Modelling

AuthorsE. Aksan*, M. Kaufmann*, O. Hilliges
In ProceedingsThe IEEE International Conference on Computer Vision (ICCV), Oct 2019
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Learning Cooperative Personalized Policies from Gaze Data

AuthorsC. Gebhardt, B. Hecox, B. van Opheusden, D. Wigdor, J. Hillis, O. Hilliges, H. Benko
In ProceedingsProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, New Orleans, LA, USA, 2019
ProjectPDFVideoVideoBibTeX
Teaser-Picture

Context-Aware Online Adaptation of Mixed Reality Interfaces

AuthorsD. Lindlbauer, A. Feit, O. Hilliges
In ProceedingsProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, New Orleans, LA, USA, 2019
ProjectPDFVideoBibTeX
Teaser-Picture

Interactive Hand Pose Estimation using a Stretch-Sensing Soft Glove

AuthorsO. Glauser, S. Wu, D. Panozzo, O. Hilliges, O. Sorkine-Hornung
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 38, Issue: 4), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

Deformation Capture via Self-Sensing Capacitive Arrays

AuthorsO. Glauser, D. Panozzo, O. Hilliges, O. Sorkine-Hornung
In ACM Transactions on Graphics (TOG) (Volume: 38, Issue: 2), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

STCN:Stochastic Temporal Convolutional Networks

AuthorsE. Aksan, O. Hilliges
In ProceedingsInternational Conference on Learning Representations (ICLR), 2019
ProjectCodePDFBibTeX
Teaser-Picture

Real-time Environment-independent Multi-view Human Pose Estimation with Aerial Vehicles

AuthorsT. Nägeli, S. Oberholzer, S. Plüss, J. Alonso-Mora, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH ASIA), Tokyo, Japan, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time

AuthorsY. Huang*, M. Kaufmann*, E. Aksan, M. Black, O. Hilliges, G. Pons-Moll
In ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), Nov 2018
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Sample Efficient Learning of Path Following and Obstacle Avoidance Behavior for Quadrotors

AuthorsS. Stevsic, T. Nägeli, J. Alonso-Mora, O. Hilliges
In IEEE Robotics and Automation Letters (Volume: 3, Issue: 4), Oct 2018
ProjectPDFVideoBibTeX
Teaser-Picture

DextrES: Wearable Haptic Feedback for Grasping in VR via a Thin Form-Factor Electrostatic Brake

AuthorsR. Hinchet*, V. Vechev*, H. Shea, O. Hilliges
In ProceedingsACM Symposium on User Interface Software and Technologies (UIST), Berlin, Germany, Oct 2018
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Deep Pictorial Gaze Estimation

AuthorsS. Park, A. Spurr, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), Munich, Germany, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Optimizing for Aesthetically Pleasing Quadrotor Camera Motion

AuthorsC. Gebhardt, S. Stevsic, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 37, Issue: 4), August 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Cross-modal deep variational hand pose estimation

AuthorsA. Spurr, J. Song, S. Park, O. Hilliges
In ProceedingsProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

Accepted as Spotlight Presentation
ProjectCodePDFVideoBibTeXVideo
Teaser-Picture

DeepWriting: Making Digital Ink Editable via Deep Generative Modeling

AuthorsE. Aksan, F. Pece, O. Hilliges
In ProceedingsSIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada, 2018

Honorable Mention Best Paper Award
ProjectCodePDFVideoBibTeX
Teaser-Picture

AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time

AuthorsS. Park, C. Gebhardt*, R. Rädle*, A. Feit, H. Vrzakova, N. Dayama, H. Yeo, C. Klokmose, A. Quigley, A. Oulasvirta, O. Hilliges
In ProceedingsSIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada, 2018
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

MagTics: Flexible and Thin Form Factor Magnetic Actuators for Dynamic and Wearable Haptic Feedback

AuthorsF. Pece*, J. Zarate*, V. Vechev, N. Besse, O. Gudozhnik, H. Shea, O. Hilliges
In ProceedingsACM Symposium on User Interface Software and Technologies (UIST), Quebec City, Canada, Oct 2017
* These two authors contributed equally to this work.

Honorable Mention Best Paper Award
ProjectPDFVideoBibTeX
Teaser-Picture

Real-time Planning for Automated Multi-View Drone Cinematography

AuthorsT. Nägeli, L. Meier, A. Domahidi, J. Alonso-Mora, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH), Los Angeles, USA, 2017
ProjectPDFBibTeX
Teaser-Picture

Thin-slicing network: A deep structured model for pose estimation in videos

AuthorsJ. Song, L. Wang, L. Van Gool, O. Hilliges
In ProceedingsProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

Accepted as oral presentation
ProjectPDFVideoVideoBibTeX
Teaser-Picture

Real-time Motion Planning for Aerial Videography with Dynamic Obstacle Avoidance and Viewpoint Optimization

AuthorsT. Nägeli, J. Alonso-Mora, A. Domahidi, D. Rus, O. Hilliges
In IEEE Robotics and Automation Letters (Volume: 2, Issue: 3), July 2017
ProjectPDFVideoBibTeX
Teaser-Picture

Interacting with soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum

AuthorsS. Wang*, J. Song*, J. Lien, I. Poupyrev, O. Hilliges
In ProceedingsProceedings of the 29th Annual Symposium on User Interface Software and Technology, 2016
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Rig Animation with a Tangible and Modular Input Device

AuthorsO. Glauser, W. Ma, D. Panozzo, A. Jacobson, O. Hilliges, O. Sorkine-Hornung
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH), Anaheim, CA, USA, Jul 2016
ProjectPDFVideoBibTeX
Teaser-Picture

Airways: Optimization-based Planning of Quadrotor Trajectories according to High-Level User Goals

AuthorsC. Gebhardt*, B. Hepp*, T. Nägeli, S. Stevsic, O. Hilliges
In ProceedingsSIGCHI Conference on Human Factors in Computing Systems, San Jose, CA, 2016
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

DefSense: Computational Design of Customized Deformable Input Devices

Authors M. Bächer, B. Hepp*, F. Pece*, P. G. Kry, B. Bickel, B. Thomaszewski, O. Hilliges
To appear in ProceedingsACM SIGCHI, San Jose, CA, USA, May 2016
*These two authors contributed equally to this work.
Project PDF Video BibTeX
Teaser-Picture

In-air Gestures Around Unmodified Mobile Devices

Authors J. Song, G. Sörös, F. Pece, S. Fanello, S. Izadi, C. Keskin, O. Hilliges
In ProceedingsACM User Interface Software and Technology Symposium, Honolulu, Hawaii, Oct. 2014
Project
Teaser-Picture

Environment-independent Formation Flight for Micro Aerial Vehicles

Authors T. Naegeli, C. Conte, A. Domahidi, M. Morari, O. Hilliges
In ProceedingsIEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, Illinois, Sept. 2014
Project PDF Video BibTeX
Teaser-Picture

Tangible and Modular Input Device for Character Articulation

Authors A. Jacobson, D. Panozzo, O. Glauser, C. Pradalier, O. Hilliges, O. Sorkine
In ProceedingsACM SIGGRAPH, Vancouver, Ca, Aug 2014
Project PDF Video BibTeX
Teaser-Picture

Type–Hover–Swipe in 96 Bytes: A Motion Sensing Mechanical Keyboard

Authors S. Taylor, C. Keskin, O. Hilliges, S. Izadi
In ProceedingsACM Conference on Human Factors in Computing Systems (ACM SIGCHI), Toronto, Ca, Apr 2014 / Best Paper Award
Project PDF Video BibTeX
Teaser-Picture

Digits: Freehand 3D Interactions Anywhere Using a Wrist-worn Gloveless Sensor

Authors D. Kim, O. Hilliges, S. Izadi, A. Butler, J. Chen, I. Oikonomidis, P. Olivier
In ProceedingsACM User Interface and Software Technologies, Boston, MA, USA, Oct 2012 / runner-up Best Demo Award
Project PDF Video BibTeX
Teaser-Picture

HoloDesk: Direct 3D Interactions with a Situated See-Through Display

Authors O. Hilliges, D. Kim, S. Izadi, M. Weiss
In ProceedingsHuman Factors in Computing Systems (ACM CHI), Austin, Tx, USA, Apr 2012
Project PDF Video BibTeX