Automatic Analysis of Macro and Micro Facial Expressions: Detection and Recognition via Machine Learning

Abstract : Facial expression analysis is an important problem in many biometric tasks, such as face recognition, face animation, affective computing and human computer interface. In this thesis, we aim at analyzing facial expressions using images and video sequences. We divided the problem into three leading parts. First, we study Macro Facial Expressions for Emotion Recognition and we propose three different levels of feature representations. Low-level feature through a Bag of Visual Word model, mid-level feature through Sparse Representation and hierarchical features through a Deep Learning based method. The objective of doing this is to find the most effective and efficient representation that contains distinctive information of expressions and that overcomes various challenges coming from: 1) intrinsic factors such as appearance and expressiveness variability and 2) extrinsic factors such as illumination, pose, scale and imaging parameters, e.g., resolution, focus, imaging, noise. Then, we incorporate the temporal dimension to extract spatio-temporal features with the objective to describe subtle feature deformations to discriminate ambiguous classes. Second, we direct our research toward transfer learning, where we aim at Adapting Facial Expression Models to New Domains and Tasks. Thus we study domain adaptation and zero shot learning for developing a method that solves the two tasks jointly. Our method is suitable for unlabelled target datasets coming from different data distributions than the source domain and for unlabelled target datasets with different label distributions but sharing the same context as the source domain. Therefore, to permit knowledge transfer between domains and tasks, we use Euclidean learning and Convolutional Neural Networks to design a mapping function that maps the visual information coming from facial expressions into a semantic space coming from a Natural Language model that encodes the visual attribute description or uses the label information. The consistency between the two subspaces is maximized by aligning them using the visual feature distribution. Third, we study Micro Facial Expression Detection. We propose an algorithm to spot micro-expression segments including the onset and offset frames and to spatially pinpoint in each image the regions involved in the micro-facial muscle movements. The problem is formulated into Anomaly Detection due to the fact that micro-expressions occur infrequently and thus leading to few data generation compared to natural facial behaviours. In this manner, first, we propose a deep Recurrent Convolutional Auto-Encoder to capture spatial and motion feature changes of natural facial behaviours. Then, a statistical based model for estimating the probability density function of normal facial behaviours while associating a discriminat- ing score to spot micro-expressions is learned based on a Gaussian Mixture Model. Finally, an adaptive thresholding technique for identifying micro expressions from natural facial behaviours is proposed. Our algorithms are tested over deliberate and spontaneous facial expression benchmarks.
Complete list of metadatas

Cited literature [378 references]  Display  Hide  Download

http://hal.univ-grenoble-alpes.fr/tel-02359665
Contributor : Dawood Al Chanti <>
Submitted on : Tuesday, November 12, 2019 - 2:50:08 PM
Last modification on : Friday, November 22, 2019 - 4:23:08 PM

File

Dawood ALCHANTI Thesis.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-02359665, version 1

Collections

Citation

Dawood Al Chanti. Automatic Analysis of Macro and Micro Facial Expressions: Detection and Recognition via Machine Learning. Image Processing [eess.IV]. UGA (Université Grenoble Alpes), 2019. English. ⟨tel-02359665⟩

Share

Metrics

Record views

98

Files downloads

109