Abstract
This course covers the core technologies required to model and simulate motions for digital humans and robotic characters. Topics include kinematic modeling, physics-based simulation, trajectory optimization, reinforcement learning, feedback control for motor skills, motion capture, data-driven motion synthesis, and ML-based generative models. They will be richly illustrated with examples.Objectives
Students will learn how to estimate human pose, shape, and motion from videos and create basic human avatars from various visual inputs. Students will also learn how to represent and algorithmically generate motions for digital characters and their real-life robotic counterparts. The lectures are accompanied by four programming assignments (written in python or C++) and a capstone project.Content
- Basic concept of 3D representations- Human body/hand models
- Human motion capture;
- Non-rigid surface tracking and reconstruction
- Neural rendering
- Optimal control and trajectory optimization
- Physics-based modeling for multibody systems
- Forward and inverse kinematics
- Rigging and keyframing
- Reinforcement learning for locomotion
Lecture Notes
Lecture recordings and slides will be available on moodlePrerequisites
Experience with python and C++ programming, numerical linear algebra, multivariate calculus and probability theory. Some background in deep learning, computer vision, physics-based modeling, kinematics, and dynamics is preferred.Administration
IMPORTANT: In case you don’t want to complete this course, please deregister by May 1st. Deregistration after the deadline will lead to fail.
| Number | 263-5806-00L |
|---|---|
| Lecturer | Prof. Dr. Stelian Coros, Prof. Dr. Siyu Tang |
| Assistants | Dongho Kang Dr. Sergey Prokudin (Head TA) Miguel Angel Zamora (Head TA) Fatemeh Zargarbashi Dr. Yan Zhang Kaifeng Zhao |
| Location and Time | Lecture: Wed 14:15-16:00 HG E 1.2 Thu 10:15-11:00 CAB G 61 Tutorial: Thu 16:15-18:00 ETF E1 |
| Moodle | https://moodle-app2.let.ethz.ch/course/view.php?id=19849 Lecture recordings and slides will be available on moodle |
| ECTS Credits | 8 |
| Exam | 40% mandatory assignments and 60% final project presentation and report |
Schedule
Lecture
| Week | Date (Wed 14-16pm) | Date (Thu 10-11am) | Topic |
|---|---|---|---|
| 01 | 22-Feb | 23-Feb | Introduction |
| 02 | 1-Mar | 2-Mar | Inverse Kinematics, Motion Capture, etc. |
| 03 | 8-Mar | 9-Mar | Optimal Control, Trajectory Optimization, etc. |
| 04 | 15-Mar | 16-Mar | Reinforcement Learning and policy gradients methods |
| 05 | 22-Mar | 23-Mar | Parametric body/hand models |
| 06 | 29-Mar | 30-Mar | Human motion estimation |
| 07 | 5-Apr | 6-Apr | Generative Models (Human motion synthesis) |
| 09 | 19-Apr | 20-Apr | Neural Fields in Visual Computing (Human avatar reconstruction) |
| 10 | 26-Apr | 27-Apr | Project presentation |
| 11 | 3-May | 4-May | No lecture |
| 12 | 10-May | 11-May | Guest lecture: Prashanth Chandran |
| 13 | 17-May | 18-May | Guest lecture: Rafael Wampfler |
| 14 | 24-May | 25-May | Guest lecture: Barbara Solenthaler |
| 15 | 31-May | 1-Jun | Project presentation |
Tutorial
| Week | Date (16pm-18pm) | Topic |
|---|---|---|
| 01 | 23-Feb | Getting started with the codebase |
| 02 | 2-Mar | Assignment 1 out |
| 03 | 9-Mar | Assignment 1 Q&A + Intro to using Euler cluster |
| 04 | 16-Mar | Assignment 2 out |
| 05 | 23-Mar | LISST tutorial/multiview |
| 06 | 30-Mar | Assignment 3 out |
| 07 | 6-Apr | PyTorch 3D & Project announcement |
| 08 | 20-Apr | Assignment 4 out & Final project overview |
| 10 | 27-Apr | Office hours for projects |
| 11 | 4-May | Office hours for projects |
| 12 | 11-May | Office hours for projects |
| 13 | 18-May | Office hours for projects |
| 14 | 25-May | Office hours for projects |
| 15 | 1-Jun | Office hours for projects |
