Abstract:

COVID has become a big problem for most universities in the world. All students were simultaneously transferred to a remote learning format, Zoom became popular for this type of events, and all classes are now provided via video conferences. Teachers have faced a huge challenge here - how to track student attendance at lectures? How to understand if a student listens carefully to a lecture, rather than just turns off the camera and goes to bed? This problem has a very serious impact on the quality of education - students have the opportunity to stop studying and at the same time stay unpunished.

However, the authors of this article see only positive aspects in the format of distance learning and believe that the quality of education could significantly improve. Each student should have a webcam and microphone. The authors want to present a program for automated scoring of presence points for students. The key purposes of the system will include several functions: to determine the student's identity; to monitor the student's gaze focus area (whether they are looking at the lecture material or enjoying their favorite streamer on the phone); to determine the number of screens, running software, etc. However, only two main modules are analyzed in detail in this paper.

The first module works directly during online classes - identification of persons using AI. The task of this module is to take a picture of a student during a lecture, connect to the internal system of the university with personal data (and photo) and confirm or deny the presence of students.

The second module is responsible for determining the student's concentration. Its main purpose is to determine, by a series of photos, where the student is currently looking - at the lecture material on the screen or in another direction. This module will be the most complicated part of the whole system because it is necessary not only to determine the direction of the student’s gaze using a non-invasive method with one camera but also to take into account the lecture material contents themself. For example, if a student is currently writing down after the teacher (and at that moment he is not looking at the screen), this is not considered a distraction.

To sum up, this article will describe the model of the system of control over the student's presence during the lecture and also describe in detail two main modules of this system - identity recognition and direction tracking - which will be based on artificial intelligence and non-invasive recording with only one camera. The approach of neural network architecture selection, collection of unlabeled data, and neural network training will be described in detail. It should be noted that at the moment the task of tracking the direction of the human gaze using only one webcam does not yet have a full implementation with sufficient accuracy, so the presented model will be experimental and the choice of the final technical solutions will be based on the best applicability after a series of tests for the task (will take into account not only the accuracy but also the overall load on the resources of the university and students).

Acknowledgement:
The discussed in this paper results were obtained in the framework of the Research Project titled "Component's digital transformation methods' fundamental research for micro- and nanosystems" (Project #0750-2020-0041) financially supported by the Russian Federation Ministry of Science and Higher Education.

Keywords:

Artificial intelligence, face recognition, eye gazement estimation, non-invasive method.