Welcome to the Semantic Morphable Models Tutorial!
In this course, you will learn how to perform a full 3D reconstruction of a face just from one single real-world image. This will be accomplished by combining a probabilistic fitting approach with explicit segmentation of an image. The main application of the segmentation is occlusion-aware face model adaptation. In this tutorial, we focus on segmenting the image into face and non-face pixels. We then adapt the face model to the face pixels only. Our approach does not rely on any training data but is built based on a generative image model, namely the 3D Morphable Model. The general concept of combined segmentation and face model adaptation is extensible in various directions. At the end of the tutorial, we will also give an outlook on possible future directions.
This tutorial is not a standalone one. It heavily builds on the knowledge you can acquire from two other tutorials:
We will use a lot of those methods without explaining them again. The focus of this tutorial is on the segmentation and occlusion-handling.
Enjoy the course!
The main idea and formal work are based on the following PhD Thesis:
The implementation and some more experimental results are derived from those publications:
Bernhard Egger, Sandro Schönborn, Andreas Schneider, Adam Kortylewski, Andreas Morel-Forster, Clemens Blumer and Thomas Vetter. Occlusion-aware 3D Morphable Models and an Illumination Prior for Face Image Analysis, International Journal of Computer Vision (IJCV), 2018
Bernhard Egger, Andreas Schneider, Clemens Blumer, Andreas Morel-Forster, Sandro Schönborn, Thomas Vetter. "Occlusion-aware 3D Morphable Face Models." British Machine Vision Conference (BMVC), September 2016
Sandro Schönborn, Bernhard Egger, Andreas Morel-Forster, Thomas Vetter. "Markov Chain Monte Carlo for Automated Face Image Analysis." International Journal of Computer Vision (IJCV), 2017
Most face images in this document are from the Labeled Faces in the Wild database.
In this tutorial, we will restrict our discussion to the method itself. For references to related work and its embedding into the wider field of model-based image analysis, occlusion-awareness and segmentation refer to above article and references therein. You can find more references to details about the material presented herein References.
Gravis Research
Department of Mathematics and Computer Science
University of Basel
Switzerland
http://gravis.cs.unibas.ch
Semantic Morphable Models Tutorial