In this tutorial, you will learn how to compute global illumination of Morphable Model objects efficiently [SSFVE17]. We use the 3D Morphable of the face as an example. Compared to local illumination, global illumination takes the whole surface of an object into account to simulate illumination effects. A global model is required to simulate the effects of self-shadowing and inter-reflections. This makes global illumination a computationally expensive task and in general requires ray casting. Creating physically plausible renderings is particularly attractive for face reconstruction within the analysis-by-synthesis approach. There, a more realistic lighting simulation benefits reconstruction. To increase rendering efficiency, we leverage that similar shapes have similar surface-light interaction. By approximating global illumination instead of simulating it with ray casting, we obtain significant performance increases at a high level of accuracy.
Approximating global illumination requires a parametrization of the full surface-light interaction. Surface-light interaction is modelled with radiance transfer functions. They define, how incoming radiance at a point is transformed to outgoing radiance. We use the concept of precomputed radiance transfer (PRT) [KSS02]. It is a linear model of radiance transfer and parameterizes surface-light interaction in the form of a matrix at every point on the shape. It allows real-time rendering of a rigid object with global illumination under arbitrary spherical harmonics environment maps. As long as the geometry stays fixed, the transfer matrices do not change, making it a very efficient in rendering global illumination of non-deforming surfaces. However, changing the geometry incurs a large performance penalty as the surface-light interaction changes.
We approximate the transfer functions by finding the linear mapping between Morphable Model shape coefficients and transfer matrices. In this tutorial you will learn how this approximation can be realized and how to use it to reconstruct faces more faithfully.
We assume, that you are familiar with the basic concepts of Morphable Models and the scalismo-faces framework. Prior knowledge about PRT is a plus, although not a necessity for this tutorial. Central ideas behind PRT are summarized in the first chapter.
The tutorial is available as a runnable jar file, where you can execute the code as you are reading it. Together with other tutorials, it can be found here: Tutorial application.
The tutorial is divided into five main chapters. We begin with a brief primer on the concept behind PRT and continue with building the MoMo-PRT model. We use the model to remove the effects of self-shadowing that is naturally present in the Morphable Model textures. With the corrected texture model, we show how to reconstruct the illumination given the shape and the target image. In the last chapter, we combine all parts to reconstruct all face parameters from a single target image.
For this tutorial you will need to download the Basel Face Model 2017 which can be found here, as it cannot be bundled with this tutorial JAR due to licensing reasons. The same also applies for the deluminated Basel Face Model which is needed for the last tutorial chapter which you can find at the same location.
The tutorial is tied to the PRT / MoMo-PRT implementations for the faces-scala framework which can be obtained from these repositories:
In order to keep the example code as simple as possible and reduce unnecessary distraction, the following packages are imported by default:
tutorial.prt._
ch.unibas.cs.gravis.tutorial._
ch.unibas.cs.gravis.tutorial.figure.InteractiveFigure
ch.unibas.cs.gravis.tutorial.figure.InteractiveRenderPanel
breeze.linalg._
scalismo.mesh._
scalismo.faces.color._
scalismo.faces.image._
scalismo.faces.mesh._
scalismo.faces.momo._
scalismo.faces.parameters._
scalismo.faces.sampling.face._
scalismo.faces.landmarks._
scalismo.faces.io._
scalismo.geometry.Vector
scalismo.utils.Random
faces.render.prt._
faces.render.prt.numerics.SH._
faces.render.prt.parametricrendering._
faces.render.prt.parametricrendering.decorators._
faces.render.prt.parametricrendering.techniques._
faces.render.prt.transfermodelling._
faces.render.prt.transfermodelling.parametermodels._
faces.render.prt.transfermodelling.transfermodels._
faces.render.prt.transfermodelling.transfermodels.linearmodel2017._
faces.render.prt.utils.SurfacePointPropertyExtension
faces.paper.model.LinearTransferModelIO
faces.paper.utils._
scala.concurrent.ExecutionContext
scala.concurrent.Await
java.util.concurrent.Executors
java.awt.event.KeyEvent
java.net.URI
java.io.File
java.io.PrintStream
java.io.OutputStream
The tutorial.prt
package provides some helper functions to facilitate repetitive tasks. Therefore some examples might look slightly different when compared with production code. Still, they do cover all relevant API of the PRT framework.
For a more detailed overview of the functions in the PRT API please refer to the documentation.
[KSS02] Kautz J, Snyder J, Sloan PP. Fast Arbitrary BRDF Shading for Low-Frequency Lighting Using Spherical Harmonics. Rendering Techniques. 2002 Jun 26;2(291-296):1.
[SSFVE17] Schneider A, Schönborn S, Frobeen L, Vetter T, Egger B. Efficient global illumination for morphable models.