Research Overview

Welcome to the AIT Lab where AI meets Interactive Technologies. We do research at the intersection of Machine Learning, Computer Vision and Human Computer Interaction (HCI). Our main mission is to develop new ways for humans to interact with complex interactive systems (computers, wearables, robots), powered by advanced algorithms and technologies in machine perception, planning and data driven user modelling. Please see research and publications for more information.

The AIT Lab, led by Prof. Dr. Otmar Hilliges, is within the Department of Computer Science at ETH Zurich. The group is jointly affiliated with the Institute of Visual Computing and Institute of Pervasive Computing.

Latest News

18.08.2020
Our paper on capturing subjective first-person view videos by drones will be presented at SIGGRAPH'20.
06.07.2020
We have 5 papers accepted at ECCV'20.
22.06.2020
We have 2 papers accepted at UIST'20.
06.01.2020
Our paper on 6D object pose estimation for assembly tasks is accepted to IEEE RA-L.
26.07.2019
5 papers from our group were accepted at ICCV'19.
01.07.2019
Sammy Christen joins the AIT lab. Welcome!
Check here for previous news.

Open PostDoc positions

We are currently looking for 2 post-docs with a background in either i) machine learning, bayesian optimisation, active learning, variational methods, deep-learning or related areas or ii) computer vision, graphics, robotics, numerical methods or physical simulation.

Excellent programming skills in C++ or other high-level language are expected and experience in Matlab, Python, OpenGL, or CUDA are beneficial. Fluent communication skills (written and presentation) in English are essential.

Positions are fully funded (via ERC starting grant) and are available to those of any nationality (some knowledge of German is beneficial but not essential). For more details see how to apply.


Latest Projects


Teaser-Picture

Capturing Subjective First-Person View Shots with Drones for Automated Cinematography

AuthorsA. Ashtari, S. Stevsic, T. Nägeli, J. Bazin, O. Hilliges
In ProceedingsACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 39, Issue: 5), Aug 2020
ProjectPDFVideoBibTeX
Teaser-Picture

Human Body Model Fitting by Learned Gradient Descent

AuthorsJ. Song*, X. Chen*, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.
ProjectPDFBibTeX
Teaser-Picture

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

AuthorsX. Chen*, Z. Dong*, J. Song, A. Geiger, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints

AuthorsA. Spurr, U. Iqbal, P. Molchanov, O. Hilliges, J. Kautz
In ProceedingsEuropean Conference on Computer Vision (ECCV), Glasgow, Scotland, 2020

Winner of HANDS2019 challenge: RGB-based 3D Hand Pose Estimation while Interacting with Objects
ProjectBibTeX
Teaser-Picture

Learning-based Region Selection for End-to-End Gaze Estimation

AuthorsX. Zhang, Y. Sugano, A. Bulling, O. Hilliges
In ProceedingsBritish Machine Vision Virtual Conference (BMVC), 2020
PDFBibTeX
Teaser-Picture

ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation

AuthorsX. Zhang, S. Park, T. Beeler, D. Bradley, S. Tang, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020

Accepted as Spotlight Presentation
ProjectPDFBibTeX

Towards End-to-end Video-based Eye-Tracking

AuthorsS. Park, E. Aksan, X. Zhang, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
ProjectPDFBibTeX
Teaser-Picture

Contact-free Nonplanar Haptics with a Spherical Electromagnet

AuthorsJ. Zarate*, T. Langerak*, B. Thomaszewski, O. Hilliges
In Proceedings2020 IEEE Haptics Symposium (HAPTICS), 2020
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation

AuthorsA. Feit*, L. Vordemann*, S. Park, C. Bérubé, O. Hilliges
In ProceedingsACM Symposium on Eye Tracking Research and Applications (ETRA), 2020
* These two authors contributed equally to this work.
ProjectCodePDFBibTeX
Teaser-Picture

Accurate Real-time 3D Gaze Tracking Using a Lightweight Eyeball Calibration

AuthorsQ. Wen, D. Bradley, T. Beeler, S. Park, O. Hilliges, J. Yong, F. Xu
In Computer Graphics Forum (Proceedings of EUROGRAPHICS), 2020
PDFBibTeX
Teaser-Picture

Learning to Assemble: Estimating 6D Poses for Robotic Object-Object Manipulation

AuthorsS. Stevsic, S. Christen, O. Hilliges
In ProceedingsIEEE Robotics and Automation Letters (Volume: 5, Issue: 2), April 2020
ProjectPDFVideoBibTeX
Teaser-Picture

Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks

AuthorsZ. He, A. Spurr, X. Zhang, O. Hilliges
In ProceedingsIEEE International Conference on Computer Vision (ICCV), Seoul, Korea, 2019
ProjectCodePDFBibTeX
Teaser-Picture

Monocular Neural Image Based Rendering with Continuous View Control

AuthorsX. Chen*, J. Song*, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019
* These two authors contributed equally to this work.
ProjectCodePDFBibTeX
Teaser-Picture

End-to-end Learning for Graph Decomposition

AuthorsJ. Song, B. Andres, M. Black, O. Hilliges, S. Tang
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019
ProjectPDFBibTeX

Few-Shot Adaptive Gaze Estimation

AuthorsS. Park*, S. De Mello*, P. Molchanov, U. Iqbal, O. Hilliges, J. Kautz
In ProceedingsInternational Conference on Computer Vision (ICCV), Seoul, Korea, 2019
* These two authors contributed equally to this work.

Accepted as Oral Presentation
ProjectCodePDFVideoBibTeX
Teaser-Picture

Structured Prediction Helps 3D Human Motion Modelling

AuthorsE. Aksan*, M. Kaufmann*, O. Hilliges
In ProceedingsThe IEEE International Conference on Computer Vision (ICCV), Oct 2019
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX

Content-Consistent Generation of Realistic Eyes with Style

AuthorsM. C. Buehler, S. Park, S. De Mello, X. Zhang, O. Hilliges
In ProceedingsInternational Conference on Computer Vision Workshops (ICCVW), Seoul, Korea, 2019

OpenEDS Image Generation Challenge Winner
ProjectPDFBibTeX
Teaser-Picture

Learning Cooperative Personalized Policies from Gaze Data

AuthorsC. Gebhardt, B. Hecox, B. van Opheusden, D. Wigdor, J. Hillis, O. Hilliges, H. Benko
In ProceedingsProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, New Orleans, LA, USA, 2019
ProjectPDFVideoVideoBibTeX
Teaser-Picture

Context-Aware Online Adaptation of Mixed Reality Interfaces

AuthorsD. Lindlbauer, A. Feit, O. Hilliges
In ProceedingsProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, New Orleans, LA, USA, 2019
ProjectPDFVideoBibTeX
Teaser-Picture

AudioTouch: Minimally Invasive Sensing of Micro-Gestures via Active Bio-Acoustic Sensing

AuthorsY. Kubo, Y. Koguchi, B. Shizuki, S. Takahashi, O. Hilliges
In ProceedingsProceedings of the 21th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2019
ProjectPDFVideoBibTeX
Teaser-Picture

Interactive Hand Pose Estimation using a Stretch-Sensing Soft Glove

AuthorsO. Glauser, S. Wu, D. Panozzo, O. Hilliges, O. Sorkine-Hornung
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 38, Issue: 4), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

Deformation Capture via Self-Sensing Capacitive Arrays

AuthorsO. Glauser, D. Panozzo, O. Hilliges, O. Sorkine-Hornung
In ACM Transactions on Graphics (TOG) (Volume: 38, Issue: 2), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

Unpaired Pose Guided Human Image Generation

AuthorsX. Chen, J. Song, O. Hilliges
In ProceedingsThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019
PDFBibTeX
Teaser-Picture

Video-based Prediction of Hand-grasp Preshaping with Application to Prosthesis Control

AuthorsL. Taverne*, M. Cognolato*, T. Bützer, R. Gassert, O. Hilliges
In ProceedingsInternational Conference on Robotics and Automation (ICRA), 2019
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Demonstration-Guided Deep Reinforcement Learning of Control Policies for Dexterous Human-Robot Interaction

AuthorsS. Christen, S. Stevsic, O. Hilliges
In ProceedingsInternational Conference on Robotics and Automation (ICRA), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

STCN:Stochastic Temporal Convolutional Networks

AuthorsE. Aksan, O. Hilliges
In ProceedingsInternational Conference on Learning Representations (ICLR), 2019
ProjectCodePDFBibTeX
Teaser-Picture

TacTiles: Dual-mode Low-power Electromagnetic Actuators for Rendering Continuous Contact and Spatial Haptic Patterns in VR

AuthorsV. Vechev*, J. Zarate*, D. Lindlbauer, R. Hinchet, H. Shea, O. Hilliges
In Proceedings2019 IEEE Virtual Reality (VR), Mar 2019
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Plan3D: Viewpoint and Trajectory Optimization for Aerial Multi-View Stereo Reconstruction

AuthorsB. Hepp, M. Niessner, O. Hilliges
In ACM Transactions on Graphics (TOG), 2018
ProjectCodePDFVideoBibTeX
Teaser-Picture

Real-time Environment-independent Multi-view Human Pose Estimation with Aerial Vehicles

AuthorsT. Nägeli, S. Oberholzer, S. Plüss, J. Alonso-Mora, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH ASIA), Tokyo, Japan, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time

AuthorsY. Huang*, M. Kaufmann*, E. Aksan, M. Black, O. Hilliges, G. Pons-Moll
In ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), Nov 2018
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Sample Efficient Learning of Path Following and Obstacle Avoidance Behavior for Quadrotors

AuthorsS. Stevsic, T. Nägeli, J. Alonso-Mora, O. Hilliges
In IEEE Robotics and Automation Letters (Volume: 3, Issue: 4), Oct 2018
ProjectPDFVideoBibTeX
Teaser-Picture

DextrES: Wearable Haptic Feedback for Grasping in VR via a Thin Form-Factor Electrostatic Brake

AuthorsR. Hinchet*, V. Vechev*, H. Shea, O. Hilliges
In ProceedingsACM Symposium on User Interface Software and Technologies (UIST), Berlin, Germany, Oct 2018
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility

AuthorsB. Hepp, D. Dey, S. Sinha, A. Kapoor, N. Joshi, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), Munich, Germany, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Deep Pictorial Gaze Estimation

AuthorsS. Park, A. Spurr, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), Munich, Germany, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Optimizing for Aesthetically Pleasing Quadrotor Camera Motion

AuthorsC. Gebhardt, S. Stevsic, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 37, Issue: 4), August 2018
ProjectPDFVideoBibTeX

Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings

AuthorsS. Park, X. Zhang, A. Bulling, O. Hilliges
In ProceedingsACM Symposium on Eye Tracking Research and Applications (ETRA), Warsaw, Poland, 2018

Best Presentation Award
ProjectCodePDFVideoBibTeX
Teaser-Picture

Cross-modal deep variational hand pose estimation

AuthorsA. Spurr, J. Song, S. Park, O. Hilliges
In ProceedingsProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

Accepted as Spotlight Presentation
ProjectCodePDFVideoBibTeXVideo
Teaser-Picture

DeepWriting: Making Digital Ink Editable via Deep Generative Modeling

AuthorsE. Aksan, F. Pece, O. Hilliges
In ProceedingsSIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada, 2018

Honorable Mention Best Paper Award
ProjectCodePDFVideoBibTeX
Teaser-Picture

AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time

AuthorsS. Park, C. Gebhardt*, R. Rädle*, A. Feit, H. Vrzakova, N. Dayama, H. Yeo, C. Klokmose, A. Quigley, A. Oulasvirta, O. Hilliges
In ProceedingsSIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada, 2018
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX