Research Overview

The AIT lab conducts research at the fore-front of human-centric computer vision. Our core research interests are algorithms and methods for the spatio-temporal understanding of how humans move within and interact with the physical world. We develop learning-based algorithms, methods and representations for human- and interaction-centric understanding of our world from videos, images and other sensor data. Application domains of interest include Augmented and Virtual Reality, Human Robot Interaction and more. Please refer to our publications for more information.

The AIT Lab, led by Prof. Dr. Otmar Hilliges, is part of the Institute for Intelligent Interactive Systems (IIS), in the Department of Computer Science at ETH Zurich.

Latest News

06.09.2022
Markos Diomataris joins the AIT lab. Welcome!
01.07.2022
Mert Albaba joins the AIT lab. Welcome!
09.08.2022
We have 1 paper accepted at TMLR. Stay tuned for more details.
09.08.2022
We have 2 papers accepted at 3DV 2022. Stay tuned for more details.
01.07.2022
We have 1 paper accepted at ECCV 2022. Stay tuned for more details.
09.05.2022
We have 1 paper accepted at SIGGRAPH 2022. Stay tuned for more details.
01.05.2022
Hsuan-I Ho and Artur Grigorev join the AIT lab. Welcome!
17.03.2022
Prof. Dr. Otmar Hilliges receives the ERC Consolidator Grant. Congratulations!
Check here for previous news.

Latest Projects


Teaser-Picture

Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors

AuthorsX. Wang*, G. Li*, Y. Kuo, M. Kocabas, E. Aksan, O. Hilliges
In ProceedingsInternational Conference on 3D Vision (3DV), 2022
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

TempCLR: Reconstructing Hands via Time-Coherent Contrastive Learning

AuthorsA. Ziani*, Z. Fan*, M. Kocabas, S. Christen, O. Hilliges
In ProceedingsInternational Conference on 3D Vision (3DV), 2022
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

SFP: State-free Priors for Exploration in Off-Policy Reinforcement Learning

AuthorsM. Bagatella, S. Christen, O. Hilliges
In ProceedingsTransactions on Machine Learning Research, 2022
ProjectCodePDFBibTeX
Teaser-Picture

EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes

AuthorsG. Li, A. Meka, F. Mueller, M. Buehler, O. Hilliges, T. Beeler
In ProceedingsACM Trans. Graph. (Volume: 41, Issue: 4), Jul 2022
ProjectPDFVideoVideoBibTeX
Teaser-Picture

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

AuthorsS. Christen, M. Kocabas, E. Aksan, J. Hwangbo, J. Song, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022
ProjectCodePDFVideoBibTeX
Teaser-Picture

gDNA: Towards Generative Detailed Neural Avatars

AuthorsX. Chen, T. Jiang, J. Song, J. Yang, M. Black, A. Geiger, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022
ProjectCodePDFVideoBibTeX
Teaser-Picture

I M Avatar: Implicit Morphable Head Avatars from Videos

AuthorsY. Zheng, V. Abrevaya, M. Bühler, X. Chen, M. Black, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022

Accepted as Oral Presentation
ProjectCodePDFVideoBibTeX
Teaser-Picture

PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence

AuthorsZ. Dong*, C. Guo*, J. Song, X. Chen, A. Geiger, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Computational Design of Kinesthetic Garments

AuthorsV. Vechev, J. Zarate, B. Thomaszewski, O. Hilliges
In ProceedingsComputer Graphics Forum, 2022
ProjectPDFVideoBibTeX
Teaser-Picture

Human Performance Capture from Monocular Video in the Wild

AuthorsC. Guo, X. Chen, J. Song, O. Hilliges
In Proceedings2021 International Conference on 3D Vision (3DV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

AuthorsK. Karunratanakul, A. Spurr, Z. Fan, O. Hilliges, S. Tang
In ProceedingsInternational Conference on 3D Vision (3DV), 2021

Accepted as Oral Presentation
ProjectPDFBibTeX
Teaser-Picture

Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation

AuthorsZ. Fan, A. Spurr, M. Kocabas, S. Tang, M. Black, O. Hilliges
In ProceedingsInternational Conference on 3D Vision (3DV), 2021

Accepted as Oral Presentation
ProjectBibTeX
Teaser-Picture

A Spatio-temporal Transformer for 3D Human Motion Prediction

AuthorsE. Aksan, M. Kaufmann, P. Cao, O. Hilliges
In ProceedingsInternational Conference on 3D Vision (3DV), 2021
ProjectCodePDFVideoBibTeX

VariTex: Variational Neural Face Textures

AuthorsM. C. Buehler, A. Meka, G. Li, T. Beeler, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

AuthorsM. Kaufmann, Y. Zhao, C. Tang, L. Tao, C. Twigg, J. Song, R. Wang, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

AuthorsX. Chen, Y. Zheng, M. Black, O. Hilliges, A. Geiger
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

Shape-aware Multi-Person Pose Estimation from Multi-view Images

AuthorsZ. Dong, J. Song, X. Chen, C. Guo, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
ProjectCodePDFVideoBibTeX
Teaser-Picture

PeCLR: Self-Supervised 3D Hand Pose Estimation from monocular RGB via Equivariant Contrastive Learning

AuthorsA. Spurr*, A. Dahiya*, X. Wang, X. Zhang, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021
* These two authors contributed equally to this work.

Accepted as Oral Presentation
ProjectCodePDFVideoBibTeX
Teaser-Picture

SPEC: Seeing People in the wild with an Estimated Camera

AuthorsM. Kocabas, C. Huang, J. Tesch, L. Müller, O. Hilliges, M. Black
In ProceedingsProceedings of the International Conference on Computer Vision 2021, Oct 2021
ProjectBibTeX
Teaser-Picture

PARE: Part Attention Regressor for 3D Human Body Estimation

AuthorsM. Kocabas, C. Huang, O. Hilliges, M. Black
In ProceedingsProceedings of the International Conference on Computer Vision 2021, Oct 2021
ProjectPDFBibTeX
Teaser-Picture

Hedgehog: Handheld Spherical Pin Array based on a CentralElectromagnetic Actuator

AuthorsA. Aline, J. Zarate, T. Langerak, V. Vechev, O. Hilliges
In ProceedingsWorld Haptics Conference, 2021

Honorable Mention - Best Paper Award
ProjectPDFVideoBibTeX

Learning Functionally Decomposed Hierarchies for Continuous Control Tasks With Path Planning

AuthorsS. Christen*, L. Jendele*, E. Aksan, O. Hilliges
In IEEE Robotics and Automation Letters (Volume: 6, Issue: 2), 2021
* These two authors contributed equally to this work.

Accepted as Oral Presentation at the Deep RL Workshop at NeurIPS
ProjectCodePDFVideoBibTeX
Teaser-Picture

Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation

AuthorsN. Vulin, S. Christen, S. Stevsic, O. Hilliges
In IEEE Robotics and Automation Letters (Volume: 6, Issue: 2), April 2021
ProjectCodePDFVideoBibTeX

Optimization-based User Support for Cinematographic Quadrotor Camera Target Framing

AuthorsC. Gebhardt, O. Hilliges
In SIGCHI Conference on Human Factors in Computing Systems, Yokohama, 2021
ProjectPDFVideoBibTeX
Teaser-Picture

Hierarchical Reinforcement Learning Explains Task Interleaving Behavior

AuthorsC. Gebhardt, A. Oulasvirta, O. Hilliges
In Computational Brain & Behavior, 2020
ProjectPDFBibTeX

CoSE: Compositional Stroke Embeddings

AuthorsE. Aksan, T. Deselaers, A. Tagliasacchi, O. Hilliges
In ProceedingsAdvances in Neural Information Processing Systems, 2020
ProjectCodePDFBibTeX

Self-Learning Transformations for Improving Gaze and Head Redirection

AuthorsY. Zheng, S. Park, X. Zhang, S. De Mello, O. Hilliges
In ProceedingsNeural Information Processing Systems (NeurIPS), 2020
ProjectCodePDFBibTeX
Teaser-Picture

Spatial Attention Improves Iterative 6D Object Pose Estimation

AuthorsS. Stevsic, O. Hilliges
In Proceedings2020 International Conference on 3D Vision (3DV), 2020
ProjectCodePDFVideoBibTeX
Teaser-Picture

Convolutional Autoencoders for Human Motion Infilling

AuthorsM. Kaufmann, E. Aksan, J. Song, F. Pece, R. Ziegler, O. Hilliges
In Proceedings2020 International Conference on 3D Vision (3DV), 2020
ProjectCodePDFVideoBibTeX
Teaser-Picture

Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback

AuthorsT. Langerak*, J. Zarate*, D. Lindlbauer, C. Holz, O. Hilliges
In ProceedingsProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, Virtual Event, USA, 2020
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Optimal Control for Electromagnetic Haptic Guidance Systems

AuthorsT. Langerak, J. Zarate, V. Vechev, D. Lindlbauer, D. Panozzo, O. Hilliges
In ProceedingsProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, Virtual Event, USA, 2020
ProjectPDFVideoBibTeX
Teaser-Picture

Learning-based Region Selection for End-to-End Gaze Estimation

AuthorsX. Zhang, Y. Sugano, A. Bulling, O. Hilliges
In ProceedingsBritish Machine Vision Virtual Conference (BMVC), 2020
PDFBibTeX
Teaser-Picture

Human Body Model Fitting by Learned Gradient Descent

AuthorsJ. Song*, X. Chen*, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

AuthorsX. Chen*, Z. Dong*, J. Song, A. Geiger, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints

AuthorsA. Spurr, U. Iqbal, P. Molchanov, O. Hilliges, J. Kautz
In ProceedingsEuropean Conference on Computer Vision (ECCV), Glasgow, Scotland, 2020

Winner of HANDS2019 challenge: RGB-based 3D Hand Pose Estimation while Interacting with Objects
ProjectBibTeX
Teaser-Picture

ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation

AuthorsX. Zhang, S. Park, T. Beeler, D. Bradley, S. Tang, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020

Accepted as Spotlight Presentation
ProjectCodePDFBibTeX

Towards End-to-end Video-based Eye-Tracking

AuthorsS. Park, E. Aksan, X. Zhang, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
ProjectCodePDFBibTeX
Teaser-Picture

Contact-free Nonplanar Haptics with a Spherical Electromagnet

AuthorsJ. Zarate*, T. Langerak*, B. Thomaszewski, O. Hilliges
In Proceedings2020 IEEE Haptics Symposium (HAPTICS), 2020
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration

AuthorsQ. Wen, D. Bradley, T. Beeler, S. Park, O. Hilliges, J. Yong, F. Xu
In Computer Graphics Forum (Proceedings of EUROGRAPHICS), 2020
PDFBibTeX
Teaser-Picture

Learning to Assemble: Estimating 6D Poses for Robotic Object-Object Manipulation

AuthorsS. Stevsic, S. Christen, O. Hilliges
In IEEE Robotics and Automation Letters (Volume: 5, Issue: 2), April 2020
ProjectPDFVideoBibTeX