Classroom Behavior Recognition Using Computer Vision: A Systematic Review.

Behavioral computing based on visual cues has become increasingly important, as it can capture and annotate teachers' and students' classroom states on a large scale and in real time. However, there is a lack of consensus on the research status and future trends of computer vision-based classroom behavior recognition. The present study conducted a systematic literature review of 80 peer-reviewed journal articles following the Preferred Reporting Items for Systematic Assessment and Meta-Analysis (PRISMA) guidelines. Three research questions were addressed concerning goal orientation, recognition techniques, and research challenges. Results showed that: (1) computer vision-supported classroom behavior recognition focused on four categories: physical action, learning engagement, attention, and emotion. Physical actions and learning engagement have been the primary recognition targets; (2) behavioral categorizations have been defined in various ways and lack connections to instructional content and events; (3) existing studies have focused on college students, especially in a natural classical classroom; (4) deep learning was the main recognition method, and the YOLO series was applicable for multiple behavioral purposes; (5) moreover, we identified challenges in experimental design, recognition methods, practical applications, and pedagogical research in computer vision. This review will not only inform the recognition and application of computer vision to classroom behavior but also provide insights for future research.