Semantic Morphable Models Tutorial

Welcome to the Semantic Morphable Models Tutorial!

In this course, you will learn how to perform a full 3D reconstruction of a face just from one single real-world image. This will be accomplished by combining a probabilistic fitting approach with explicit segmentation of an image. The main application of the segmentation is occlusion-aware face model adaptation. In this tutorial, we focus on segmenting the image into face and non-face pixels. We then adapt the face model to the face pixels only. Our approach does not rely on any training data but is built based on a generative image model, namely the 3D Morphable Model. The general concept of combined segmentation and face model adaptation is extensible in various directions. At the end of the tutorial, we will also give an outlook on possible future directions.

This tutorial is not a standalone one. It heavily builds on the knowledge you can acquire from two other tutorials:

We will use a lot of those methods without explaining them again. The focus of this tutorial is on the segmentation and occlusion-handling.

Enjoy the course!

Chapters

  1. Motivation
  2. Segmentation Basics
  3. Semantic Morphable Model
  4. Inference / EM-like algorithm
  5. Initialization / robust illumination estimation
  6. Occlusion-aware model adaptation
  7. Outlook / future work

References

The main idea and formal work are based on the following PhD Thesis:

The implementation and some more experimental results are derived from those publications:

Most face images in this document are from the Labeled Faces in the Wild database.

In this tutorial, we will restrict our discussion to the method itself. For references to related work and its embedding into the wider field of model-based image analysis, occlusion-awareness and segmentation refer to above article and references therein. You can find more references to details about the material presented herein References.

Authors

Gravis Research
Department of Mathematics and Computer Science
University of Basel
Switzerland
http://gravis.cs.unibas.ch


Semantic Morphable Models Tutorial