A Cascaded YOLOv8s-ResNet-18 Framework for Facial Expression Recognition

School of Computer Science and Technology, Harbin Institute of Technology, Weihai, Shandong, China

^* This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

In intelligent human-computer interaction and monitoring device applications, facial expression recognition is an important technique. The recognition accuracy can directly influence the success rate of practical systems and the user experience. However, in uncontrolled scenes, the background may be cluttered and the face position may be unknown; under this condition, a single convolutional neural network is easily affected by noise, and the recognition accuracy can drop clearly. Therefore, a model with better resistance to interference in complex environments has practical value. Based on the public FER2013 dataset, this study first trained ResNet-18 as the basic model; then the pre-trained YOLOv8s model was connected with the trained ResNet-18 classifier, so a combined framework of detection first and classification later was built. To imitate real complex environments, this study also constructed a synthetic complex-scene test set by processing images from the original test set in several ways. On the original test set, ResNet-18 performed normally, but on the complex-scene test set, the accuracy of the single model was only 23.3%. After the YOLOv8s detector was added, the combined model reached an end-to-end accuracy of 51.99% under the same complex-scene setting; this result was much better than that of the single model. The experimental results show that the pre-detection module can improve the model’s ability to resist background interference in complex environments. This work provides a useful reference framework and possible improvement direction for real-time facial expression recognition systems in practical applications.