Matsuno SHOGO (松野 省吾)

In this paper, we propose a method for constructing a gaze direction estimation model using Vision Transformer (ViT) as a fundamental technology for developing input interfaces using eye movement. Eye movement measurement methods using machine learning enable highly accurate estimation but require many computational resources. Therefore, in this paper, we consider a calibration-free and lightweight gaze direction identification model with implementation in gaze input interfaces in mind. The proposed method constructs a gaze direction classification model by fine-tuning a large-scale pre-trained model of ViT using the constructed dataset. The training dataset was constructed by extracting the face region as a still image for each frame from a video image captured by a webcam and then focusing on the area near the eyeball. In addition, we experimented to evaluate the performance of the gaze direction estimation model constructed using the proposed method. As a result of the experiment, the accuracy rate and macro average F-value of the proposed method were approximately 19.0 points higher than the conventional method, and we confirmed the overall improvement of the classification performance under calibration-free.

We have developed blink measurement methods that can be applied to input interfaces. To use the eye-blinking information as an input trigger, it is necessary to automatically classify blink types into voluntary and involuntary. A method for blink type classification using a three-dimensional convolutional neural network (3D-CNN) has been proposed. This classification method takes a short image sequence of the periocular area and classifies the blink type. We previously reported on several performance-improving methods that can be applied to this 3D-CNN. Since our classification using 3D-CNN outputs classification results in units of video frames, multiple types of classification results could be mixed together during a period of a single blinking motion. To address this problem, we employ a correction method to calculate the mode value as a representative value for the consecutive blink period. This paper proposes a correction method to improve accuracy based on limiting the aggregation range of the mode to a reliable portion. The evaluation experiment resulted in 97.9% accuracy and 94.0% F-score in the classification results for each short image sequence for 10 subjects. Then, 97.5% accuracy and 97.3% F-score were obtained for the accuracy of blink type classification.