Recognition of Learners’ Cognitive States using Facial Expressions in E-learning Environments

Technological developments in e-learning systems provide new opportunities for students to enhance academic growth and improve access to education. E-learning is on the rise because it has advantages over conventional learning. The need for time is a faster and simpler learning experience. The spread of the coronavirus disease (COVID-19) pandemic has resulted in school closures around the world. More than one billion students are out of the classroom worldwide. As a consequence, education has taken on a new shape, with a major increase in e-learning, whereby teaching is carried out online and on digital platforms. The scope of this research is to identify the facial expressions of the students and then link these expressions to the cognitive states. This paper presents a hybrid-CNN model to recognize a learner’s cognitive state using the manually engineered features and features extracted from the convolutional neural network. In addition, the performance of the model is compared with the manual feature extraction method and CNN methods separately. The proposed method trained and tested with the spontaneous database(DAiSEE) created exclusively for the e-learning environment, and with the state-of-the-art datasets such as JAFFE and CK+. The model has achieved 53.4%, 71.4%, and 99.95% respectively.