Computer Science

CLC number: TP751

Deep Learning-Based Lecture Posture Evaluation

基于深度学习的讲课姿态评估

¹ Key Laboratory of Multidisciplinary Management and Control of Complex Systems of Anhui Higher Education Institutes, Anhui University of Technology, Maanshan 243002, Anhui, China
² School of Microelectronics and Data Science, Anhui University of Technology, Maanshan 243032, Anhui, China

^† Corresponding author. E-mail: zt9877@163.com

Received: 8 August 2023

Abstract

Computer vision, a scientific discipline enables machines to perceive visual information, aims to supplant human eyes in tasks encompassing object recognition, localization, and tracking. In traditional educational settings, instructors or evaluators evaluate teaching performance based on subjective judgment. However, with the continuous advancements in computer vision technology, it becomes increasingly crucial for computers to take on the role of judges in obtaining vital information and making unbiased evaluations. Against this backdrop, this paper proposes a deep learning-based approach for evaluating lecture posture. First, feature information is extracted from various dimensions, including head position, hand gestures, and body posture, using a human pose estimation algorithm. Second, a machine learning-based regression model is employed to predict machine scores by comparing the extracted features with expert-assigned human scores. The correlation between machine scores and human scores is investigated through experiment and analysis, revealing a robust overall correlation (0.642 0) between predicted machine scores and human scores. Under ideal scoring conditions (100 points), approximately 51.72% of predicted machine scores exhibited deviations within a range of 10 points, while around 81.87% displayed deviations within a range of 20 points; only a minimal percentage of 0.12% demonstrated deviations exceeding the threshold of 50 points. Finally, to further optimize performance, additional features related to bodily movements are extracted by introducing facial expression recognition and gesture recognition algorithms. The fusion of multiple models resulted in an overall average correlation improvement of 0.022 6.

摘要

计算机视觉作为一门如何让机器"看"的科学,旨在让计算机代替人类的眼睛来对目标进行识别、定位和跟踪等。在过去,评委老师通常根据自己的判断来评估老师的讲课水平。随着计算机视觉技术的发展,用计算机代替评委来获取关键信息并进行判断具有非常重要的意义。基于上述背景,本文提出了一种基于深度学习的讲课姿态评估方法。首先通过人体姿态估计算法从人体的头部、手部、躯干等多个维度上提取特征信息,然后利用机器学习的回归模型对提取的特征和人工分预测机器分,最后分析了机器分与人工分之间的相关度以及分数偏差。实验结果表明,机器预测分与人工分整体相关度达到0.642 0,表现出强相关的水平。在满分100分的情况下, 机器预测分和人工分的分差在10分以内和20分以内的占比分别达到51.72%和81.87%,分差50分以上的情况下仅占0.12%。为了优化效果,引入了人脸表情和手势识别算法来提取更多的人体特征,多模型融合下的整体相关度提升了0.022 6。