VLG | Computer Vision and Learning Group

Abstract

This course covers the core technologies required to model and simulate motions for digital humans. The curriculum includes human body modeling, human motion capture, data-driven human motion synthesis, and ML-based generative models. Each topic will be extensively illustrated with examples to provide a comprehensive understanding of the subject matter.

Objectives

Students will learn how to estimate human pose, shape, and motion from videos and create basic human avatars from various visual inputs. Students will also learn how to represent and algorithmically generate motions for digital characters. The lectures are accompanied by exercise sessions and a capstone project.

Content

- Basic concepts of 3D representations
- Human body/hand models
- Human motion capture;
- Neural rendering
- Transformers
- Generative models for digital humans

Lecture Notes

Lecture recordings and slides will be available on moodle

Prerequisites

Experience with python and C++ programming, numerical linear algebra, multivariate calculus and probability theory. Solid background in deep learning, computer vision, physics-based modeling, kinematics, and dynamics is preferred.

Administration

IMPORTANT: ~~The deadline to cancel/deregister from the course is March 19. Deregistration after the deadline will lead to fail.~~

Since Digital Humans changes to include a session exam this year, the deregistration deadline for the course has been updated to be the deregistration deadline for session exams. Students now have until 28^th July 2024, to deregister from the exam without receiving a failed result.

Number	263-5806-00L
Location and Time	Lecture: Tue 14:15-17:00 CAB G 61 Tutorial: Thu 16:15-18:00 ETF E1
Moodle	Lecture recordings and slides will be available on moodle
ECTS Credits	8
Examination	50% session exam (written 90 minutes) and 50% final project presentation and report

Schedule

Lecture

Week	Date (Wed 14-17pm)	Topic
01	20-Feb	Introduction
02	27-Feb	Human body models
03	5-Mar	Model fitting
04	12-Mar	Project presentation
05	19-Mar	Neural representation
06	26-Mar	Neural representation
07		Easter break
08	9-Apr	Transformer
09	16-Apr	Generative Models
10	23-Apr	Project presentation
11	30-Apr	Generative Models
12	7-May	Guest lecture
13	14-May	Guest lecture
14	21-May	Guest lecture
15	28-May	Project presentation

Tutorial

Week	Date (Thu 16-18pm)	Topic
01	22-Feb	Project introduction
02	29-Feb	Euler cluster introduction
03	7-Mar	Project discussion
04	14-Mar	Exercise (body model)
05	21-Mar	Exercise (model fitting)
06	28-Mar
07	4-Apr
08	11-Apr	Exercise (Neural field)
09	18-Apr	Exercise (Transformer)
10	25-Apr	Project discussion
11	2-May	Exercise (VAE)
12	9-May
13	16-May	Exercise (Diffusion)
14	23-May
15	30-May