This paper presents two novel handheld projector systems for indoor pervasive computing spaces. These projection-based devices are “aware” of their environment in ways not demonstrated previously. They offer both spatial awareness, where the system infers location and orientation of the device in 3D space, and geometry awareness, where the system constructs the 3D structure of the world around it, which can encompass the user as well as other physical objects, such as furniture and walls.

Previous work in this area has predominantly focused on infrastructure-based spatial-aware handheld projection and interaction. Our prototypes offer greater levels of environment awareness, but achieve this using two opposing approaches; the first infrastructure-based and the other infrastructure-less sensing. We highlight a series of interactions including direct touch, as well as in-air gestures, which leverage the shadow of the user for interaction.

We describe the technical challenges in realizing these novel systems; and compare them directly by quantifying their location tracking and input sensing capabilities.

Accompanying Video

ACM Digital Library

Published at

ACM User Interface and Software Technologies (UIST), 2012

Best Paper Award

Project Links


@inproceedings{Molyneaux:2012:IEH:2367310.2367327, author = {Molyneaux, David and Izadi, Shahram and Kim, David and Hilliges, Otmar and Hodges, Steve and Cao, Xiang and Butler, Alex and Gellersen, Hans}, title = {Interactive Environment-aware Handheld Projectors for Pervasive Computing Spaces}, booktitle = {Proceedings of the 10th International Conference on Pervasive Computing}, series = {Pervasive'12}, year = {2012}, isbn = {978-3-642-31204-5}, location = {Newcastle, UK}, pages = {197--215}, numpages = {19}, url = {http://dx.doi.org/10.1007/978-3-642-31205-2_13}, doi = {10.1007/978-3-642-31205-2_13}, acmid = {2367327}, publisher = {Springer-Verlag}, address = {Berlin, Heidelberg}, keywords = {geometry and spatial awareness, handheld projection, interaction}, }