Biography

I'm a Postdoctoral researcher at ETH Zurich in the Advanced Interactive Technologies lab. . Briefly, I am interested in how to incorporate structured information into deep learning pipeline, either semantically or geometrically. Application-wise, I am mainly working on hand-pose/body-pose-shape/human-motion estimation and also learning based view synthesis and 6D pose estimation. If you are interested in Semester/Master projects, please directly contact me by email. Especially, if you are a fan of football, check this link for thesis.

Awards

• 3DV best paper award 2017
• Qualcomm innovation fellowship finalist 2015
• Swisscom innovation award 2014
• Birkigt Scholarship of ETH Zürich 2013
• National Scholarship 2009/2010

Publications

gDNA: Towards Generative Detailed Neural Avatars

AuthorsX. Chen, T. Jiang, J. Song, J. Yang, M. Black, A. Geiger, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022

ProjectCodePDFVideoBibTeX

---

PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence

AuthorsZ. Dong*, C. Guo*, J. Song, X. Chen, A. Geiger, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022
* These two authors contributed equally to this work.

ProjectPDFVideoBibTeX

---

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

AuthorsS. Christen, M. Kocabas, E. Aksan, J. Hwangbo, J. Song, O. Hilliges
In ProceedingsComputer Vision and Pattern Recognition (CVPR), 2022

ProjectPDFBibTeX

---

Human Performance Capture from Monocular Video in the Wild

AuthorsC. Guo, X. Chen, J. Song, O. Hilliges
In Proceedings2021 International Conference on 3D Vision (3DV), 2021

ProjectCodePDFVideoBibTeX

---

Shape-aware Multi-Person Pose Estimation from Multi-view Images

AuthorsZ. Dong, J. Song, X. Chen, C. Guo, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021

ProjectCodePDFVideoBibTeX

---

EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

AuthorsM. Kaufmann, Y. Zhao, C. Tang, L. Tao, C. Twigg, J. Song, R. Wang, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), 2021

ProjectCodePDFVideoBibTeX

---

Convolutional Autoencoders for Human Motion Infilling

AuthorsM. Kaufmann, E. Aksan, J. Song, F. Pece, R. Ziegler, O. Hilliges
In Proceedings2020 International Conference on 3D Vision (3DV), 2020

ProjectCodePDFVideoBibTeX

---

Human Body Model Fitting by Learned Gradient Descent

AuthorsJ. Song*, X. Chen*, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.

ProjectCodePDFVideoBibTeX

---

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

AuthorsX. Chen*, Z. Dong*, J. Song, A. Geiger, O. Hilliges
In ProceedingsEuropean Conference on Computer Vision (ECCV), 2020
* These two authors contributed equally to this work.

ProjectCodePDFVideoBibTeX

---

Monocular Neural Image Based Rendering with Continuous View Control

AuthorsX. Chen*, J. Song*, O. Hilliges
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019
* These two authors contributed equally to this work.

ProjectCodePDFBibTeX

---

End-to-end Learning for Graph Decomposition

AuthorsJ. Song, B. Andres, M. Black, O. Hilliges, S. Tang
In ProceedingsInternational Conference on Computer Vision (ICCV), Oct 2019

ProjectPDFBibTeX

---

Unpaired Pose Guided Human Image Generation

AuthorsX. Chen, J. Song, O. Hilliges
In ProceedingsThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019

CodePDFBibTeX

---

Cross-modal deep variational hand pose estimation

AuthorsA. Spurr, J. Song, S. Park, O. Hilliges
In ProceedingsProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

Accepted as Spotlight Presentation

ProjectCodePDFVideoBibTeXVideo

---

Learning human motion models for long-term predictions

AuthorsP. Ghosh, J. Song, E. Aksan, O. Hilliges
In Proceedings2017 International Conference on 3D Vision (3DV), 2017

Best Paper Award

ProjectCodePDFVideoBibTeX

---

Thin-slicing network: A deep structured model for pose estimation in videos

AuthorsJ. Song, L. Wang, L. Van Gool, O. Hilliges
In ProceedingsProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

Accepted as oral presentation

ProjectPDFVideoVideoBibTeX

---

Two-Stream SR-CNNs for Action Recognition in Videos

Authors Y. Wang, J. Song, L. Wang, L. Van Gool, O. Hilliges
In ProceedingsBMVC, York, UK, 2016
Code and Models are on GitHub.

PDF BibTeX

---

Interacting with Soli: Exploring Fine-Grained Dynamic Gesture Recognition in the Radio-Frequency Spectrum

Authors S. Wang*, J. Song*, J. Lien, I. Poupyrev, O. Hilliges
In ProceedingsACM User Interface Software and Technology Symposium, Tokyo, Japan, 2016
Code and dataset is on GitHub.
*The first two authors contributed equally to this work.

Project PDF Video BibTeX

---

Joint Estimation of 3D Hand Position and Gestures from Monocular Video for Mobile Interaction

Authors J. Song, F. Pece, G. Sörös, M. Koelle, O. Hilliges
In ProceedingsACM SIGCHI, Seoul, South Korea, Apr. 2015

Project PDF Video BibTeX

---

In-air Gestures Around Unmodified Mobile Devices

Authors J. Song, G. Sörös, F. Pece, S. Fanello, S. Izadi, C. Keskin, O. Hilliges
In ProceedingsACM User Interface Software and Technology Symposium, Honolulu, Hawaii, Oct. 2014

Project PDF Video BibTeX

---

Student Theses

Past Student Projects

• Personalized neural avatar reconstruction. Chen Guo, 2021, MA (now PHD at ETH)
• Multi-person pose estimation. Zijian Dong, 2021, MA (now PHD at ETH)
• Pose guided human image generation. Xu Chen, 2019, MA (now PHD at ETH)
• Skeleton based human motion modelling. Partha Gosh, 2017, MA (now PHD at MPI tuebingen)
• Skeleton based human action recognition. Sephora Madjiheurem,2017, MA (now PHD at UCL)
• RNN based gesture recognition for Radio-frequency spectrum. Saiwen Wang, 2016, MA (now Apple)
• CNN based action recognition in videos. Yifan Wang, 2015, MA (now PHD at Disney/ETHZ)

Teaching Assistant

• Visual Computing (D-INFK) — 2019,2018,2017;
• Machine Perception (D-INFK) — 2020,2019,2018;
• Parallel Programming (D-INFK) — 2017,2016;
• Machine Learning (D-INFK) — 2016,2015;
• User Interface Engineering (D-INFK) — 2015,2014;