You are here

Warning message

Attention! This event has already passed.

Facial analysis and synthesis

Monday, 8 May, 2006 - 17:00
Campus: Brussels Humanities, Sciences & Engineering campus
auditorium P. Janssens
Ilse Ravyse
phd defence

Communication of the face motion is an important aspect of the human interaction. Its
automatic extraction from an image sequence is a challenging task due to the complexity of
the face gestures. It requires the measurement and modeling of all spatial-temporal
semantical properties of the face in the scene. In this dissertation, we propose the estimation
of the face motion by the aid of a virtual face model that contains the related a-priori face
knowledge. The sublime combination of computer vision, mechanical laws and computer
graphics allows not only to analyze the rigid and nonrigid face motion, but at the same time it
is capable to re-create the expressions on the 3-dimensional face model. Furthermore, the
next generation video-compression standard MPEG4 intends to represent a face movie via a
similar parameterized face model and foresees an efficient storage and transmission for it.

In this context, we have developed a Facial Analysis and Synthesis Scheme that
accomplishes the extraction of the parameters of the 3-dimensional face motion from a 2-
dimensional recording of an actor’s performance. The natural modelling of facial color, shape
and apparent motion permits the retrieval of the valuable face gesture information lost in the
3- to 2-dimensional conversion inherent to the imaging process. This inverse problem has
been solved at different levels:

- Face extraction and head tracking exploit the facial color and spatial properties. In the first
image frame of a sequence, an ellipsoid head region is found by a personalized skin color
detection and a shape-based face verification. Then, to follow this region in time, a
tracking algorithm has been developed that models the joint probability of the location and
the color of the detected face and that can deal with scale changes. This new method
achieves a better robustness than the color-histogram trackers.

- Automatic gesture analysis is provided by the image filtering of a mathematical
morphology scale-space. The information about the eye or mouth opening, contained in
the local scale images, is assembled in shape control parameter. It is a fast technique of
that can observe the evolution of the facial features.

- Scene calibration prepares the face model geometry to supports the motion. Using the
semantic correspondences between the face image and the model, a camera calibration
positions the model in the 3-dimensional face scene, and a structure calibration makes the
model adaptations to person’s looks. The selection of the face morphing algorithm which
does not distort the face model has been based on a new sensitivity study of the radial
basis function interpolation.

- The estimation of the 3-dimensional facial expressions is an ill-posed problem due to the
high degree of deformation of the face surface. It cannot be solved without imposing
constraints on the face motion. In this thesis, a new estimation approach integrates a
natural motion model using the muscle forces caused by displacements (implemented
using a finite element model). We took into account the motion projection by linking the
optical flow measurement with the modelled scene flow in a nonlinear least-squares
formulation, and by giving a physical motivation to the regularization. Smooth
deformations of the face model have been obtained.

- Graphically represented face animations benefit from control rules which can be easily
applied to a face model. We have realized this by the dynamic application of the MPEG4
face animation parameters and their calculation from the extracted displacements.

Animating the face model with the extracted parameters, results in a faithful reproduction of
the gestures. Applications in the emerging media which convey this synthetic content via a
virtual human are numerous, amongst them are the interactive television, games, telepresence,
news-reading on internet and intuitive user-interfaces.