Early Explorations using KNN to Classify Emotions in Virtual Reality based on Heart Rate (HR) and Electrodermography (EDG)

¹ Faculty of Computing and Informatics, Universiti Malaysia Sabah, Kota Kinabalu, Sabah, Malaysia
² Creative Advanced Machine Intelligence Research Centre, Faculty of Computing and Informatics, Universiti Malaysia Sabah, Kota Kinabalu, Sabah, Malaysia
³ Evolutionary Computing Laboratory, Faculty of Computing and Informatics, Universiti Malaysia Sabah, Kota Kinabalu, Sabah, Malaysia

^* Corresponding author: jtwteo@ums.edu.my, aaronfb91@gmail.com, jmountstephens@ums.edu.my

Abstract

To detect multimodal emotions using Virtual Reality (VR), this research demonstrates the findings and results of using a KNN Classifier by merging Heart Rate and Electrodermography signals. The participants in the study were shown 360-degree videos using a VR headset to elicit their emotional reactions. A wearable that measures skin activity and pulse rate in real time was used to record their emotional response. The experiment had a total of 30 participants, and the KNN classifier was used to classify intra-subject data. With the HR combined with EDG signals paired with KNN as the classifier, the study’s 30 participants’ data went through intra-subject classification where 11 out of 30 participants achieved a peak accuracy of 100%. These findings show that by combining HR and EDG signals, KNN may be used as the classifier to produce highly accurate results. This study’s possible applications include VR rehabilitation, gaming, and entertainment.