User Interface Engineering - SS '17

An in-depth introduction to the core concepts of intelligent user-interfaces. The course primarily deals with machine analysis of human non-verbal behavior and its applications to human-computer, human-robot, and computer-mediated human-human interaction. Methods involve machine learning, deep learning and model based optimization.


eDoz Course Nr.
O. Hilliges, F. Pece
E. Aksan and C. Gebhardt
Thu 10:00 - 12:00, NO C 6
Thu 13:00 - 15:00, NO C 6
Office hours
coordinate via e-mail:


Project (Kaggle) and ProjectB (Chalearn) are launched.
Second assignment is launched.
Azure platform for the exercises is finally online. Check your email for instructions
First assignment is launched.
Course website online

Learning Objectives

Students will learn about fundamental aspects of modern intelligent user interfaces. After completing the course students will have acquired theoretical and practical knowledge about the most important problems in machine understanding of human behavior and how to leverage such understanding in the design of intelligent user-facing technologies.

The core competency acquired through this course is a solid foundation in machine learning and deep-learning algorithms to process and interpret human input into computing systems. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multi-modal data, among others. Furthermore, students will be able to leverage models of human behavior in optimization based (algorithmic) design of user interfaces.


Wk.Date ContentSlides Extra Material
1 24.02.

Introduction to class contents & admin

2 02.03.
ML for HCI Pt. I

Linear Models

slides slides (annotated)
ipynb ipynb (Azure)
3 09.03.
ML for HCI Pt. II

Non-linear SVM & Decision Trees

slides slides (annotated)
4 16.03.
ML for HCI Pt. III

Ensemble Methods

slides slides (annotated) pptx (wth videos)
5 23.03.
Dynamic input


slides slides (annotated) hmm-solution hmm-ipynb (Azure)
6 30.03.
Deep Learning

Backprop Algorithm

slides slides (annotated)
7 06.04.
Deep Learning


slides slides (annotated)
8 13.04.
Deep Learning

Sequence Modelling

slides slides (annotated)
9 20.04.
No Class (Easter)
10 27.04.
Case Study: Programm Committee
11 04.05.
User Modelling

Basics of Perception

12 11.05.
Motor system & text input

Fitts' law, language & touch models

13 18.05.
Computational UI Design

Algorithmic design of UIs

slides keyopt-ipynb solution
14 25.05.
No Class (Ascension Day)


There will be 3 exercises (2 homework assignments and 1 case study) and one multi-week project. The exercises will constitute 40 % of the final grade. Assignments have to be completed individually. It is ok to discuss with your team members but you have to write your own code.

Exercise sheets and solutions will only be accessible from within the ETH network.

Exercise Assignment Results Due date
Exercise 1 Slides Jupyter Notebooks Assignment Page

Register at Kaggle:

Grades 22.03.2017
Exercise 2 Slides Code Assignment Page

Register at Kaggle:

Slides Code
Grades 26.04.2017
Exercise 3 Program Committee Review Form Grades Reviews 27.04.2017
Project Slides Project Codes Project Page (Kaggle)

Register at Kaggle:

ProjectB Codes ProjectB Page (ChaLearn)
Grades 16.06.2017

Case Study

We will do one in class case study, simulating a program committee meeting. This is a mandatory and graded part of the course requirements.


The performance assessment is an oral exam conducted during the examiniation session (Jul-Aug). It will constitute 60 % of the final grade.