User Interface Engineering - FS '13

This course provides an in-depth introduction to the core concepts of user interface engineering such as user interface technologies, input/output devices, and algorithms to process human input into computing systems. The course focuses on current topics in UI design, in particular post-desktop interaction, mobile device interaction, augmented and mixed reality, and advanced sensor and output technologies.


eDoz Course Nr.
O. Hilliges
Tobias Nägeli, Karthik Sheshadri, and Liu Zhiyong
Wed 13:15 - 15:00, HG E 33.3
Wed 15:15 - 16:00, CNB D101
Office hours
coordinate via e-mail:
ECTS credits
ECTS credits: 4 credits


The catalogue of potential topics for the midterm is now online.
The midterm exam will be on Nov 6th during the first hour of the class.
Anonymous Feedback form is now online.
Please contact the TA's for administrative or organisational questions.
Note new location for the Exercises CNB D101 (Pervasive Lab)
The exercises (with presence) will only start in the second week of the term on 25.9.2013. However there will be a reading assignment in Week 1.

Learning Objectives and Course Content

By the end of the course students will be able to design and implement functional interactive systems, that sense user input and extract semantic meaning from low-level sensor data, and respond to the sensed input with synthesized output (e.g., graphics, audio). Throughout the course students will master the following constituting elements:

  1. Predict which sensor is best suited to implement a given design.
    1. Describe working principles of various sensor technologies.
    2. Analyze the advantages and disadvantages of different sensors in respect to the intended use.
  2. Compare, choose and implement different algorithms for user input recognition (i.e., gesture recognition, stroke recognition)
  3. Decide how to map user input to the manipulation of digital information and on-screen output.
  4. Read and critique descriptions of interactive systems in research papers.
  5. Present and reason about design choices made during the implementation of an interactive system.


Wk.Date Content Slides Reading
1 18.09.

Introduction to class contents, HCI research field, admin

2 25.09.
Input Fundamentals I

Buttons, switches, sliders. Keyboards & Mice, Textinput

3 02.10.
Interactive Surfaces & Gestural Interfaces

Overview of post-WIMP Interfaces and Interaction Techniques (image processing for vision based touch screens)

4 09.10.
No Class (ACM UIST)

Work on homework assignments / Project ideas

No Slides
5 16.10.
Camera based input I

Techniques & algorithms for touch detection on interactive surfaces (tracking, filtering etc.)

slides KF_intro
6 23.10.
Advanced Input Devices & Techniques

Touchscreen technologies. Resistive / Capacitive

slides slides_interactive_seq
7 30.10.
Camera based input II

Techniques & algorithms for the analysis of motion and shape based interaction (mostly Optical Flow)

8 06.11.
Midterm & Presentation of Project Ideas

Attendance required!

No Slides
9 13.11.
Gesture recognition I

2D Gesture and stroke recognition.

10 20.11.
Gesture recognition II

Advanced techniques for gesture recognition.

11 27.11.
Intro to body and hand pose estimation

Basic introduction to body (and particularly hand) pose estimation. Random Forests.

slides PPT with embedded videos
12 04.12.
Hand pose estimation & HMMs

Advanced algorithms for hand pose estimation. HMMs for gesture recognition.

13 11.12.
Augmented Reality

Overview of Augmented Reality research field. Camera tracking, scene reconstruction.

14 18.12.
Review & Project presentations

Exercise groups present their projects prototypes

No Slides


There will be 5 programming (and some hardware building) assignments. The homework constitutes 40% of the final grade. Assignments have to be completed individually. It is ok to discuss with your team members but you have to write your own code. You will have to be able to explain your implementation line-by-line.

The assignment sheets will be available on Monday before each weeks Exercise slot (Wed 15-16). Please read these before you come to the Exercise and ask questions if something is unclear. Exercise sheets and solutions will only be accessible from within the ETH network.

Exercise Assignment Due date
Exercise 1 Homework1 Homework_supp Skel_calib Skel_img Solution code 03/10/2013
Exercise 2 Homework2 Skeleton code Solution code 17/10/2013
Exercise 3 Homework3 Skeleton code(hardwr) Skeleton code(vid) Solution PDF 25/10/2013
Exercise 4 Homework4 Skeleton code Solution Solution code 13/11/2013


There will be a midterm pen-and-paper exam which makes up 20% of your final grade. There is no final exam but you will have to present your project results.

New: The midterm will take place on Nov 6th 2013 during the first hour of the lecture in HG E 33.3

New: Mid term topic catalogue is now online.

The midterm points are available here.


In lieu of a final exam, students will complete a course project. The project is an opportunity to participate in the conceptualization, design and actual implementation of an interactive HCI system. Ideally, your project will advance the state of the art in a particular area of HCI. Office hours with the instructor/TAs are according to the usual schedule, or by email.


At the end of the semester (Dec 18th), project groups will present their prototypes during the lecture. Each group must also submit a written report describing their work. For your report, please follow the template here. Please adhere to a page limit of 6 pages in SIGCHI two column format. You must also submit your source code together with the report. All deliverables are due on Dec 18th. Your report should address the following points about your project:


Group Members Project Description Final Presentation
Oliver Voggenreiter, Roland Meyer Uberpong pdf
Stefan Messmer, Julia Strasser, Mauro Bieg Combining Touch and Depth Input for new User Control pdf