Abstract:
As a new basic technique for visual perception and cognition study,eye tracking technology is now a hot research topic in computer science,psychology,human-computer interaction,and other fields.Traditional eye-tracking systems mostly use CCD/CMOS cameras,and the tracking frequency is limited to about 100 Hz,which makes it difficult to meet the demands of high-frequency applications such as gazebased identity verification and mental disorder diagnosis.In contrast,event cameras can capture fast and erratic eye movements with sub-microsecond temporal resolution so that are more adaptable and flexible.At present,the most advanced event-based eye-tracking algorithms available are model-based.They are not robust to the subject diversity and sensor noise so that are difficult to be implemented in domains that require high accuracy and portability.Therefore,in this paper,we propose and implement a novel hybrid frame-event eye-tracking system that combine the benefits of deep learning techniques not requiring calibration and not relying on infrared light sources with the high resolution of event cameras.The system extracts key points of eyes from the frame and event images,fed them into a ConvLSTM-based pupil detection model to determine the pupil center coordinates.Eye-movement feature vectors are obtained by combining the extracted eye point corner,pupil center,and position information from the two.A neural network model is then used to classify and establish the gaze-point mapping relationship to achieve high-frequency eye movement tracking.