Research Overview

Welcome to the AIT Lab where AI meets Interactive Technologies. We do research at the intersection of Machine Learning, Computer Vision and Human Computer Interaction (HCI). Our main mission is to develop new ways for humans to interact with complex interactive systems (computers, wearables, robots), powered by advanced algorithms and technologies in machine perception, planning and data driven user modelling. Please see research and publications for more information.

The AIT Lab, led by Prof. Dr. Otmar Hilliges, is within the Department of Computer Science at ETH Zurich. The group is jointly affiliated with the Institute of Visual Computing and Institute of Pervasive Computing.

Latest News

28.06.2019
We have two papers accepted conditionally at ACM UIST'19.
28.06.2019
We will be presenting three papers at SIGGRAPH'19. One on quadrotor motion planning for 3D reconstruction, one on deformation capture via capacitive arrays, and one on applying this to hand pose estimation using a soft printable glove.
01.04.2019
Xu Chen joins the AIT lab. Welcome!
15.02.2019
Our paper on generative modeling of time-series is accepted to ICLR'19.
16.10.2018
Prof. Dr. Otmar Hilliges serves as General Chair at ISMAR'18
01.10.2018
Thomas Langerak joins the AIT lab. Welcome!
29.09.2018
One paper on real-time human pose estimation from sparse IMUs and one paper on a drone-based motion capture system (Flycon) accepted at SIGGRAPH Asia'18.
01.09.2018
David Lindlbauer joins the AIT lab. Welcome!
01.09.2018
Anna Feit joins the AIT lab. Welcome!

Open PostDoc positions

We are currently looking for 2 post-docs with a background in either i) machine learning, bayesian optimisation, active learning, variational methods, deep-learning or related areas or ii) computer vision, graphics, robotics, numerical methods or physical simulation.

Excellent programming skills in C++ or other high-level language are expected and experience in Matlab, Python, OpenGL, or CUDA are beneficial. Fluent communication skills (written and presentation) in English are essential.

Positions are fully funded (via ERC starting grant) and are available to those of any nationality (some knowledge of German is beneficial but not essential). For more details see how to apply.


Latest Projects


Teaser-Picture

Interactive Hand Pose Estimation using a Stretch-Sensing Soft Glove

AuthorsO. Glauser, S. Wu, D. Panozzo, O. Hilliges, O. Sorkine-Hornung
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 38, Issue: 4), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

Deformation Capture via Self-Sensing Capacitive Arrays

AuthorsO. Glauser, D. Panozzo, O. Hilliges, O. Sorkine-Hornung
In ACM Transactions on Graphics (TOG) (Volume: 38, Issue: 2), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

Video-based Prediction of Hand-grasp Preshaping with Application to Prosthesis Control

AuthorsL. Taverne*, M. Cognolato*, T. Bützer, R. Gassert, O. Hilliges
In ProceedingsInternational Conference on Robotics and Automation (ICRA), 2019
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Demonstration-Guided Deep Reinforcement Learning of Control Policies for Dexterous Human-Robot Interaction

AuthorsS. Christen, S. Stevsic, O. Hilliges
In ProceedingsInternational Conference on Robotics and Automation (ICRA), 2019
ProjectPDFVideoBibTeX
Teaser-Picture

STCN:Stochastic Temporal Convolutional Networks

AuthorsE. Aksan, O. Hilliges
In ProceedingsInternational Conference on Learning Representations (ICLR), 2019
ProjectCodePDFBibTeX
Teaser-Picture

TacTiles: Dual-mode Low-power Electromagnetic Actuators for Rendering Continuous Contact and Spatial Haptic Patterns in VR

AuthorsV. Vechev*, J. Zarate*, D. Lindlbauer, R. Hinchet, H. Shea, O. Hilliges
In Proceedings2019 IEEE Virtual Reality (VR), Mar 2019
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Plan3D: Viewpoint and Trajectory Optimization for Aerial Multi-View Stereo Reconstruction

AuthorsB. Hepp, M. Niessner, O. Hilliges
In ACM Transactions on Graphics (TOG), 2018
ProjectCodePDFVideoBibTeX
Teaser-Picture

Real-time Environment-independent Multi-view Human Pose Estimation with Aerial Vehicles

AuthorsT. Nägeli, S. Oberholzer, S. Plüss, J. Alonso-Mora, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH ASIA), Tokyo, Japan, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time

AuthorsY. Huang*, M. Kaufmann*, E. Aksan, M. Black, O. Hilliges, G. Pons-Moll
In ACM Transactions on Graphics, (Proc. SIGGRAPH Asia), Nov 2018
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Sample Efficient Learning of Path Following and Obstacle Avoidance Behavior for Quadrotors

AuthorsS. Stevsic, T. Nägeli, J. Alonso-Mora, O. Hilliges
In IEEE Robotics and Automation Letters (Volume: 3, Issue: 4), Oct 2018
ProjectPDFVideoBibTeX
Teaser-Picture

DextrES: Wearable Haptic Feedback for Grasping in VR via a Thin Form-Factor Electrostatic Brake

AuthorsR. Hinchet*, V. Vechev*, H. Shea, O. Hilliges
In ProceedingsACM Symposium on User Interface Software and Technologies (UIST), Berlin, Germany, Oct 2018
* These two authors contributed equally to this work.
ProjectPDFVideoBibTeX
Teaser-Picture

Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility

AuthorsB. Hepp, D. Dey, S. Sinha, A. Kapoor, N. Joshi, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), Munich, Germany, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Deep Pictorial Gaze Estimation

AuthorsS. Park, A. Spurr, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), Munich, Germany, 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Optimizing for Aesthetically Pleasing Quadrotor Camera Motion

AuthorsC. Gebhardt, S. Stevsic, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (Volume: 37, Issue: 4), August 2018
ProjectPDFVideoBibTeX
Teaser-Picture

Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings

AuthorsS. Park, X. Zhang, A. Bulling, O. Hilliges
In ProceedingsACM Symposium on Eye Tracking Research and Applications (ETRA), Warsaw, Poland, 2018

Best Presentation Award
ProjectCodePDFVideoBibTeX
Teaser-Picture

Cross-modal Deep Variational Hand Pose Estimation

AuthorsA. Spurr, J. Song, S. Park, O. Hilliges
In ProceedingsCVPR, Salt Lake City, USA, 2018

Accepted as Spotlight Presentation
ProjectCodePDFVideoBibTeXVideo
Teaser-Picture

DeepWriting: Making Digital Ink Editable via Deep Generative Modeling

AuthorsE. Aksan, F. Pece, O. Hilliges
In ProceedingsSIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada, 2018

Honorable Mention Best Paper Award
ProjectCodePDFVideoBibTeX
Teaser-Picture

AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time

AuthorsS. Park, C. Gebhardt*, R. Rädle*, A. Feit, H. Vrzakova, N. Dayama, H. Yeo, C. Klokmose, A. Quigley, A. Oulasvirta, O. Hilliges
In ProceedingsSIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada, 2018
* These two authors contributed equally to this work.
ProjectCodePDFVideoBibTeX
Teaser-Picture

Learning Human Motion Models for Long-term Predictions

AuthorsP. Ghosh, J. Song, E. Aksan, O. Hilliges
In Proceedings3D Vision (3DV), 2017 International Conference on, 2017

Best Paper Award
ProjectCodePDFVideoBibTeX
Teaser-Picture

MagTics: Flexible and Thin Form Factor Magnetic Actuators for Dynamic and Wearable Haptic Feedback

AuthorsF. Pece*, J. Zarate*, V. Vechev, N. Besse, O. Gudozhnik, H. Shea, O. Hilliges
In ProceedingsACM Symposium on User Interface Software and Technologies (UIST), Quebec City, Canada, Oct 2017
* These two authors contributed equally to this work.

Honorable Mention Best Paper Award
ProjectPDFVideoBibTeX
Teaser-Picture

Real-time Planning for Automated Multi-View Drone Cinematography

AuthorsT. Nägeli, L. Meier, A. Domahidi, J. Alonso-Mora, O. Hilliges
In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH), Los Angeles, USA, 2017
ProjectPDFBibTeX
Teaser-Picture

Guiding InfoGAN with Semi-Supervision

AuthorsA. Spurr, E. Aksan, O. Hilliges
In ProceedingsECML PKDD, Skopje, Macedonia, 2017
ProjectCodePDFBibTeX
Teaser-Picture

Thin-Slicing Network: A Deep Structured Model for Pose Estimation in Videos

AuthorsJ. Song, L. Wang, L. Van Gool, O. Hilliges
In ProceedingsCVPR, Hawaii, USA, 2017

Accepted as oral presentation
ProjectPDFVideoVideoBibTeX