Abstract

Human digitalization is required in many applications, such as AR/VR, robotics, games, and social networking. The course covers core techniques and fundamental tools necessary for perceiving and modeling humans. The main topics include human body modeling, human appearance and motion modeling, and human-​scene interaction capture and modeling.

Objectives

After attending this course, students will be able to implement basic systems to estimate human pose, shape, and motion from videos; furthermore, students will be able to create basic human avatars from various visual inputs.

Content

We will focus on all aspects of 3D human capture, modelling, and synthesis, including
⁃ Basic concept of 3D representations
⁃ Human body models;
⁃ Human motion capture;
⁃ Non-​rigid surface tracking and reconstruction;
⁃ Neural rendering

Lecture Notes

Slides

Literature

Computer Vision: Algorithms and applications by Richard Szeliski.
Deep Learning: by Goodfellow, Bengio, and Courville

Prerequisites

This is an advanced lecture for learning to model and synthesize 3D humans. We assume you have basic knowledge of computer vision, deep learning, and computer graphics; a solid understanding of linear algebra, probability, and calculus.
The following courses are highly recommended as a prerequisite
visual computing, computer vision, and deep learning.

Administration

Number 263-​5906-00L
Lecturer Prof. Dr. Siyu Tang
Assistants Dr. Yan Zhang (Head TA)
Dr. Sergey Prokudin
Qianli Ma
Shaofei Wang
Location Tuesday, CAB G 51
Moodle Link https://moodle-​app2.let.ethz.ch/course/view.php?id=17072
Lecture recordings and slides will be available on moodle
ECTS Credits 5
Exam Graded semester performance

Schedule

Week Date (14-​16pm) Topic
01 22-​Feb Introduction
02 1-​Mar Body models
03 8-​Mar Individual discussion with TAs and supervisers
04 15-​Mar Project presentation
05 22-​Mar Hand models
06 29-​Mar Human pose estimation
07 5-​Apr Rendering
08 12-​Apr Neural Rendering
10 26-​Apr Project presentation
11 3-​May Neural body models
12 10-​May Non-​rigid surface tracking
13 17-​May Motion capture and motion matching
14 24-​May Motion modeling
15 31-​May Project final presentation 1

Week Date (14pm-​15pm) Topic
01 24-​Feb Project overview
02 3-​Mar no tutorial
03 10-​Mar SMPL Tutorial
04 17-​Mar no tutorial
05 24-​Mar paper presentation
06 31-​Mar paper presentation
07 7-​Apr paper presentation
08 14-​Apr paper presentation
10 28-​Apr paper presentation
11 5-​May paper presentation
12 12-​May paper presentation
13 19-​May paper presentation
14 26-​May paper presentation
15 2-​Jun Project final presentation 2