Neural Network Learns to Recognize Emotions, Personality Traits and Ambivalence at Once
Researchers at HSE University, working with colleagues from the St. Petersburg Federal Research Center of the Russian Academy of Sciences and Sber’s Center for Practical Artificial Intelligence, have developed a neural network that can recognize emotions, assess personality traits and detect ambivalence, or uncertainty and inconsistency in a person’s behavior.
The model analyzes video, voice and speech content, as well as facial expressions, gaze, posture and gestures, and performs all three tasks simultaneously. This is more accurate than analyzing speech or facial expressions alone.
The system was trained on HSE University’s Charisma supercomputer using several different datasets. According to Dmitry Ryumin, associate professor at HSE University’s School of Information Technology, Physics and Technology in St. Petersburg, the approach makes it possible to analyze different aspects of human behavior in relation to one another.
The approach improved emotion recognition by about 6% and personality-trait assessment by 3.2%. On data the model had not seen before, emotion-recognition accuracy increased by 13.7%. The researchers plan to apply the technology in education and user-support services.
