Affective computing : unconventional emotions recognition

During my study, I participated to innovative projects such as this one. This project aim at finding novels and explorings solutions to implement an incremental learning model including an active learning solution for emotion recognition (CFEE, Compound Facial Expressions of Emotion)

This project was proposed by the “Learning Data Robotics” (LDR) research laboratory of ESIEA. The supervision is provided by Lionel Prevost, University Professor, Director of the LDR (Learning, Data & Robotics) laboratory, and Khadija Slimani, Doctor, member of the LDR.

Logo ESIEA
Logo LDR

Source : ESIEA

Emotion recognition is a well-known issue in artificial intelligence; affective computing aims to analyze human emotional states using video data to determine emotional states through non-verbal, paraverbal, and verbal cues.

The state of the art in this field mainly focuses on basic emotions (fear, anger, joy, surprise, sadness, and disgust as defined by Ekman). However, in many contexts, individuals rarely express these types of emotions and instead express complex emotions such as confidence or pride. Moreover, the interpretation of these emotions is not universal. The goal of our project is to improve classification by adding emotions to the model during the learning process.

    
        
Example of basic emotions

Source : MULTIMODAL SYSTEM FOR FACIAL EMOTION RECOGNITION BASED ON DEEP LEARNING

The laboratory explores emotion recognition solutions in these particular contexts:

  • Pedagogical interactions to create an intelligent and affective tutoring system capable of best supporting the student in their learning;
  • Public speaking (debate, job interview) to help the individual control their non-verbal communication.

The objectives of the project are:

  • Conduct a state-of-the-art review on incremental learning;
  • Develop a semi-automatic annotation tool;
  • Develop incremental models in classes and data.

To see the results.


State of the Art

One of the first tasks before exploring potential directions is conducting a state of the art. This section explains the key concepts explored during the literature review.

Exploration and Research

This section is dedicated to the various explorations and ideas we developed to implement an incremental solution for emotion recognition.

Implementation

The final experimental implementation is based on incremental models from the scikit-learn library. These models use Action Units (AU) extracted from CFEE dataset images, as well as feature vectors from pre-trained models.