KinectFusion: real-time dynamic 3D surface reconstruction and interaction

Shahram Izadi, Richard A. Newcombe, David Kim, Otmar Hilliges, David Molyneaux, Steve Hodges, Pushmeet Kohli, Jamie Shotton, Andrew J. Davison, Andrew Fitzgibbon

Teaser-Picture

Abstract

KinectFusion enables a user holding and moving a standard Kinect camera to rapidly create detailed 3D reconstructions of an indoor scene. Only the depth data from Kinect is used to track the 3D pose of the sensor and reconstruct, geometrically precise, 3D models of the physical scene in real-time.

The capabilities of KinectFusion, as well as the novel GPU-based pipeline are described in full. Uses of the core system for low-cost handheld scanning, and geometry-aware augmented reality and physics-based interactions are shown.

Novel extensions to the core GPU pipeline demonstrate object segmentation and user interaction directly in front of the sensor, without degrading camera tracking or reconstruction. These extensions are used to enable real-time multi-touch interactions anywhere, allowing any planar or non-planar reconstructed physical surface to be appropriated for touch.

Accompanying Video

ACM Digital Library



Published at

ACM SIGGRAPH, 2011

Project Links

Video Paper

Bibtex

@inproceedings{Izadi:2011:KRD:2037826.2037857, author = {Izadi, Shahram and Newcombe, Richard A. and Kim, David and Hilliges, Otmar and Molyneaux, David and Hodges, Steve and Kohli, Pushmeet and Shotton, Jamie and Davison, Andrew J. and Fitzgibbon, Andrew}, title = {KinectFusion: Real-time Dynamic 3D Surface Reconstruction and Interaction}, booktitle = {ACM SIGGRAPH 2011 Talks}, series = {SIGGRAPH '11}, year = {2011}, isbn = {978-1-4503-0974-5}, location = {Vancouver, British Columbia, Canada}, pages = {23:1--23:1}, articleno = {23}, numpages = {1}, url = {http://doi.acm.org/10.1145/2037826.2037857}, doi = {10.1145/2037826.2037857}, acmid = {2037857}, publisher = {ACM}, address = {New York, NY, USA}, }