Biography

I'm a PHD candidate at ETH Zurich in the Advanced Interactive Technologies lab. . Briefly, I am interested in how to incorporate structured information into deep learning pipeline, either semantically or geometrically. Application-wise, I am mainly working on hand-pose/body-pose-shape/human-motion estimation and also learning based view synthesis and 6D pose estimation. If you are interested in Semester/Master projects, please directly contact me by email.

Awards

• 3DV best paper award 2017
• Qualcomm innovation fellowship finalist 2015
• Swisscom innovation award 2014
• Birkigt Scholarship of ETH Zürich 2013

Publications

Convolutional Autoencoders for Human Motion Infilling

AuthorsM. Kaufmann, E. Aksan, J. Song, F. Pece, R. Ziegler, O. Hilliges
In Proceedings2020 International Conference on 3D Vision (3DV), 2020

ProjectPDFVideoBibTeX

---

Human Body Model Fitting by Learned Gradient Descent

AuthorsJ. Song*, X. Chen*, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.

ProjectPDFVideoBibTeX

---

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

AuthorsX. Chen*, Z. Dong*, J. Song, A. Geiger, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.

ProjectPDFVideoBibTeX

---

Monocular Neural Image Based Rendering with Continuous View Control

AuthorsX. Chen*, J. Song*, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019
* These two authors contributed equally to this work.

ProjectCodePDFBibTeX

---

End-to-end Learning for Graph Decomposition

AuthorsJ. Song, B. Andres, M. Black, O. Hilliges, S. Tang
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019

ProjectPDFBibTeX

---

Unpaired Pose Guided Human Image Generation

AuthorsX. Chen, J. Song, O. Hilliges
In ProceedingsThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019

PDFBibTeX

---

Cross-modal deep variational hand pose estimation

AuthorsA. Spurr, J. Song, S. Park, O. Hilliges
In ProceedingsProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

Accepted as Spotlight Presentation

ProjectCodePDFVideoBibTeXVideo

---

Learning human motion models for long-term predictions

AuthorsP. Ghosh, J. Song, E. Aksan, O. Hilliges
In Proceedings2017 International Conference on 3D Vision (3DV), 2017

Best Paper Award

ProjectCodePDFVideoBibTeX

---

Thin-slicing network: A deep structured model for pose estimation in videos

AuthorsJ. Song, L. Wang, L. Van Gool, O. Hilliges
In ProceedingsProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

Accepted as oral presentation

ProjectPDFVideoVideoBibTeX

---

Two-Stream SR-CNNs for Action Recognition in Videos

Authors Y. Wang, J. Song, L. Wang, L. Van Gool, O. Hilliges
In ProceedingsBMVC, York, UK, 2016
Code and Models are on GitHub.

PDF BibTeX

---

Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum

Authors S. Wang*, J. Song*, J. Lien, I. Poupyrev, O. Hilliges
In ProceedingsACM User Interface Software and Technology Symposium, Tokyo, Japan, 2016
Code and dataset is on GitHub.
*The first two authors contributed equally to this work.

Project PDF Video BibTeX

---

Joint Estimation of 3D Hand Position and Gestures from Monocular Video for Mobile Interaction

Authors J. Song, F. Pece, G. Sörös, M. Koelle, O. Hilliges
In ProceedingsACM SIGCHI, Seoul, South Korea, Apr. 2015

Project PDF Video BibTeX

---

In-air Gestures Around Unmodified Mobile Devices

Authors J. Song, G. Sörös, F. Pece, S. Fanello, S. Izadi, C. Keskin, O. Hilliges
In ProceedingsACM User Interface Software and Technology Symposium, Honolulu, Hawaii, Oct. 2014

Project PDF Video BibTeX

---

Student Theses

Past Student Projects

• Pose Guided Human Image Generation. Xu Chen, 2019, MA (now PHD at ETH)
• Skeleton based human motion modelling. Partha Gosh, 2017, MA (now PHD at MPI tuebingen)
• Skeleton based human action recognition. Sephora Madjiheurem,2017, MA (now PHD at UCL)
• RNN based gesture recognition for Radio-frequency spectrum. Saiwen Wang, 2016, MA (now Apple)
• CNN based action recognition in videos. Yifan Wang, 2015, MA (now PHD at Disney/ETHZ)

Teaching Assistant

• Visual Computing (D-INFK) — 2019,2018,2017;
• Machine Perception (D-INFK) — 2020,2019,2018;
• Parallel Programming (D-INFK) — 2017,2016;
• Machine Learning (D-INFK) — 2016,2015;
• User Interface Engineering (D-INFK) — 2015,2014;