Automatic Human Facial Expression Recognition Based on Integrated Classifier From Monocular Video with Uncalibrated Camera

¹ Systems Engineering Institute, School of Electronic & Information Engineering, Xi’an JiaoTong University, 28 Xian-Ning West Road, Xi’an city, Shaanxi Province, 710049, PR China
² State key laboratory for systems engineering, Xi’an Jiaotong University, 28 Xian-Ning West Road, Xi’an city, Shaanxi Province, 710049, PR China
³ Research Institute of Computer Software & Theory, Department of Computer Science & Technology, School of Electronics & Information Engineering, Xi’an Jiaotong University, 28 Xian-Ning West Road, Xi’an city, Shaanxi Province, 710049, PR China

Abstract

An automatic recognition framework for human facial expressions from a monocular video with an uncalibrated camera is proposed. The expression characteristics are first acquired from a kind of deformable template, similar to a facial muscle distribution. After associated regularization, the time sequences from the trait changes in space-time under complete expressional production are then arranged line by line in a matrix. Next, the matrix dimensionality is reduced by a method of manifold learning of neighborhood-preserving embedding. Finally, the refined matrix containing the expression trait information is recognized by a classifier that integrates the hidden conditional random field (HCRF) and support vector machine (SVM). In an experiment using the Cohn–Kanade database, the proposed method showed a comparatively higher recognition rate than the individual HCRF or SVM methods in direct recognition from two-dimensional human face traits. Moreover, the proposed method was shown to be more robust than the typical Kotsia method because the former contains more structural characteristics of the data to be classified in space-time