In this chapter, we would like to summarize options to further extend our semantic morphable model framework and what kind of ideas could be integrated. Most of those research directions result out of this thesis: Bernhard Egger "Semantic Morphable Models" PhD Thesis, University of Basel, 2017.
Besides from using a likelihood that changes for facial regions we could also include separate models like a separate model for the eye region. The explicit segmentation would then define the responsibilities of the different models to explain parts of the image.
The segmentation we used is based on integer labels. Uncertainty in the labels is not used during our EM-style inference. However, a soft-segmentation has several benefits for diverse applications (e.g. image manipulation).
Recent deep learning methods reach reasonable segmentation results by bottom-up cues only. Such methods could be integrated into the framework and used to guide the segmentation. Both, the model adaptation and the segmentation are mainly focussed on our strong prior on facial appearance and could profit from those powerful bottom-up methods.
Semantic Morphable Models Tutorial | Outlook / future work