Gaze tracking is increasingly becoming an essential component in Augmented and Virtual Reality. Modern gaze tracking al gorithms are heavyweight; they operate at most 5 Hz on mobile processors despite that near-eye cameras comfortably operate at a r eal-time rate ($>$ 30 Hz). This paper presents a real-time eye tracking algorithm that, on average, operates at 30 Hz on a mobile processor, achieves \ang{0.1}--\ang{0.5} gaze accuracies, all the while requiring only 30K parameters, one to two orders of magn itude smaller than state-of-the-art eye tracking algorithms. The crux of our algorithm is an Auto~ROI mode, which continuously pr edicts the Regions of Interest (ROIs) of near-eye images and judiciously processes only the ROIs for gaze estimation. To that end, we introduce a novel, lightweight ROI prediction algorithm by emulating an event camera. We discuss how a software emulation of events enables accurate ROI prediction without requiring special hardware. The code of our paper is available at https://github.com/horizon-research/edgaze.
In this paper, we consider the problem of tracking the eye-gaze of individuals while they engage in reading. Particularly, we develop ways to accurately track the line being read by an individual using commercially available eye tracking devices. Such an approach will enable futuristic functionalities such as comprehension evaluation, interest level detection, and user-assisting applications like hands-free navigation and automatic scrolling. Existing commercial eye trackers provide an estimated location of the eye-gaze fixations every few milliseconds. However, this estimated data is found to…
In this paper a review is presented of the research on eye gaze estimation techniques and applications, that has progressed in diverse ways over the past two decades. Several generic eye gaze use-cases are identified: desktop, TV, head-mounted, automotive and handheld devices. Analysis of the literature leads to the identification of several platform specific factors that influence gaze tracking accuracy. A key outcome from this review is the realization of a need to develop standardized methodologies for performance evaluation of gaze tracking systems and achieve consistency in their specific…
In this paper, we present an analysis of eye gaze patterns pertaining to visual cues in augmented reality (AR) for head-mounted displays (HMDs). We conducted an experimental study involving a picking and assembly task, which was guided by different visual cues. We compare these visual cues along multiple dimensions (in-view vs. out-of-view, static vs. dynamic, sequential vs. simultaneous) and analyze quantitative metrics such as gaze distribution, gaze duration, and gaze path distance. Our results indicate that visual cues in AR significantly affect eye gaze patterns. Specifically, we show tha…
Eye movements hold information about human perception, intention, and cognitive state. We propose a novel eye movement simulator that i) probabilistically simulates saccade movements as gamma distributions considering different peak velocities and ii) models smooth pursuit onsets with the sigmoid function. Additionally, it is capable of producing velocity and two-dimensional gaze sequences for static and dynamic scenes using saliency maps or real fixation targets. Our approach is also capable of simulating any sampling rate, even with uctuations. The simulation is evaluated against publicly av…
Technical progress in hardware and software enables us to record gaze data in everyday situations and over long time spans. Among a multitude of research opportunities, this technology enables visualization researchers to catch a glimpse behind performance measures and into the perceptual and cognitive processes of people using visualization techniques. The majority of eye tracking studies performed for visualization research is limited to the analysis of gaze distributions and aggregated statistics, thus only covering a small portion of insights that can be derived from gaze data. We argue th…
In recent times, there has been significant interest in the machine recognition of human emotions, due to the suite of applications to which this knowledge can be applied. A number of different modalities, such as speech or facial expression, individually and with eye gaze, have been investigated by the affective computing research community to either classify the emotion (e.g. sad, happy, angry) or predict the continuous values of affective dimensions (e.g. valence, arousal, dominance) at each moment in time. Surprisingly after an extensive literature review, eye gaze as a unimodal input to a…
Webcam eye tracking for the collection of gaze data in the context of user studies is convenient - it can be used in remote tests where participants do not need special hardware. The approach has strong limitations, especially regarding the motion-free nature of the test persons during data recording and the quality of the gaze data obtained. Our study with 52 participants shows that usable eye tracking data can be obtained with commercially available webcams in a remote setting. However, a high drop off rate must be considered, which is why we recommend a high over-recruitment of 150%. We als…
While being able to read with screen magnifiers, low vision people have slow and unpleasant reading experiences. Eye tracking has the potential to improve their experience by recognizing fine-grained gaze behaviors and providing more targeted enhancements. To inspire gaze-based low vision technology, we investigate the suitable method to collect low vision users' gaze data via commercial eye trackers and thoroughly explore their challenges in reading based on their gaze behaviors. With an improved calibration interface, we collected the gaze data of 20 low vision participants and 20 sighted co…
Speech has been a widely used modality in the field of affective computing. Recently however, there has been a growing interest in the use of multi-modal affective computing systems. These multi-modal systems incorporate both verbal and non-verbal features for affective computing tasks. Such multi-modal affective computing systems are advantageous for emotion assessment of individuals in audio-video communication environments such as teleconferencing, healthcare, and education. From a review of the literature, the use of eye gaze features extracted from video is a modality that has remained la…
We present GazeBaseVR, a large-scale, longitudinal, binocular eye-tracking (ET) dataset collected at 250 Hz with an ET-enabled virtual-reality (VR) headset. GazeBaseVR comprises 5,020 binocular recordings from a diverse population of 407 college-aged participants. Participants were recorded up to six times each over a 26-month period, each time performing a series of five different ET tasks: (1) a vergence task, (2) a horizontal smooth pursuit task, (3) a video-viewing task, (4) a self-paced reading task, and (5) a random oblique saccade task. Many of these participants have also been recorded…
One of the main challenges of gaze-based interactions is the ability to distinguish normal eye function from a deliberate interaction with the computer system, commonly referred to as 'Midas touch'. In this paper we propose, EyeTAP (Eye tracking point-and-select by Targeted Acoustic Pulse) a hands-free interaction method for point-and-select tasks. We evaluated the prototype in two separate user studies, each containing two experiments with 33 participants and found that EyeTAP is robust even in presence of ambient noise in the audio input signal with tolerance of up to 70 dB, results in a fas…
In this paper, we describe the results of replicating one of our studies from two years ago which compares two techniques for transferring content across multiple screens in VR. Results from the previous study have shown that a combined gaze and touch input can outperform a bimanual touch-only input in terms of task completion time, simulator sickness, task load and usability. Except for the simulator sickness, these findings could be validated by the replication. The difference with regards to simulator sickness and variations in absolute scores of the other measures could be explained by a d…
SurgGaze: Implicit Calibration for Accurate Gaze Analysis in Operating Rooms with Wearable Eyetrackers
Computer Science and Engineering, University of Michigan（密歇根大学计算机科学与工程系）；Michigan Institute for Data & AI in Society, University of Michigan（密歇根大学数据与人工智能社会研究所）；School of Information, University of Michigan（密歇根大学信息学院）；Medical School, University of Michigan（密歇根大学医学院）
Gaze-based virtual keyboards provide an effective interface for text entry by eye movements. The efficiency and usability of these keyboards have traditionally been evaluated with conventional text entry performance measures such as words per minute, keystrokes per character, backspace usage, etc. However, in comparison to the traditional text entry approaches, gaze-based typing involves natural eye movements that are highly correlated with human brain cognition. Employing eye gaze as an input could lead to excessive mental demand, and in this work we argue the need to include cognitive load a…
Pie menus are suggested as powerful tool for eye gaze based text entry among various interfaces developed so far. If pie menus are used with multiple depth layers then multiple saccades are required per selection of item, which is inefficient because it consumes more time. Also dwell time selection method is limited in performance because higher dwell time suffers from inefficiency while lower one from inaccuracy. To overcome problems with multiple depth layers and dwell time, we designed Quickpie, an interface for eye gaze based text entry with only one depth layer of pie menu and selection b…
Near distances are overestimated in virtual reality, and far distances are underestimated, but an explanation for these distortions remains elusive. One potential concern is that whilst the eye rotates to look at the virtual scene, the virtual cameras remain static. Could using eye-tracking to change the perspective of the virtual cameras as the eye rotates improve depth perception in virtual reality? This paper identifies 14 distinct perspective distortions that could in theory occur from keeping the virtual cameras fixed whilst the eye rotates in the context of near-eye displays. However, th…
Comparing mobile eye tracking data from multiple participants without information about areas of interest (AOIs) is challenging because of individual timing and coordinate systems. We present a technique, the gaze spiral, that visualizes individual recordings based on image content of the stimulus. The spiral layout of the slitscan visualization is used to create a compact representation of scanpaths. The visualization provides an overview of multiple recordings even for long time spans and helps identify and annotate recurring patterns within recordings. The gaze spirals can also serve as gly…
Affective computing research traditionally focused on labeling a person's emotion as one of a discrete number of classes e.g. happy or sad. In recent times, more attention has been given to continuous affect prediction across dimensions in the emotional space, e.g. arousal and valence. Continuous affect prediction is the task of predicting a numerical value for different emotion dimensions. The application of continuous affect prediction is powerful in domains involving real-time audio-visual communications which could include remote or assistive technologies for psychological assessment of su…
Technological Institute of the Philippines（菲律宾理工学院）
Eye-GUIDE is an assistive communication tool designed for the paralyzed or physically impaired people who were unable to move parts of their bodies especially people whose communications are limited only to eye movements. The prototype consists of a camera and a computer. Camera captures images then it will be send to the computer, where the computer will be the one to interpret the data. Thus, Eye-GUIDE focuses on camera-based gaze tracking. The proponent designed the prototype to perform simple tasks and provides graphical user interface in order the paralyzed or physically impaired person c…
Appearance-based gaze estimation methods that only require an off-the-shelf camera have significantly improved but they are still not yet widely used in the human-computer interaction (HCI) community. This is partly because it remains unclear how they perform compared to model-based approaches as well as dominant, special-purpose eye tracking equipment. To address this limitation, we evaluate the performance of state-of-the-art appearance-based gaze estimation for interaction scenarios with and without personal calibration, indoors and outdoors, for different sensing distances, as well as for …
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
3D gaze information is important for scene-centric attention analysis but accurate estimation and analysis of 3D gaze in real-world environments remains challenging. We present a novel 3D gaze estimation method for monocular head-mounted eye trackers. In contrast to previous work, our method does not aim to infer 3D eyeball poses but directly maps 2D pupil positions to 3D gaze directions in scene camera coordinate space. We first provide a detailed discussion of the 3D gaze estimation task and summarize different methods, including our own. We then evaluate the performance of different 3D gaze…
With eye tracking being increasingly integrated into virtual and augmented reality (VR/AR) head-mounted displays, preserving users' privacy is an ever more important, yet under-explored, topic in the eye tracking community. We report a large-scale online survey (N=124) on privacy aspects of eye tracking that provides the first comprehensive account of with whom, for which services, and to what extent users are willing to share their gaze data. Using these insights, we design a privacy-aware VR interface that uses differential privacy, which we evaluate on a new 20-participant dataset for two p…
University of Stuttgart（斯图加特大学）；University of Southampton（南安普顿大学）
World-wide-web, with the website and webpage as the main interface, facilitates the dissemination of important information. Hence it is crucial to optimize them for better user interaction, which is primarily done by analyzing users' behavior, especially users' eye-gaze locations. However, gathering these data is still considered to be labor and time intensive. In this work, we enable the development of automatic eye-gaze estimations given a website screenshots as the input. This is done by the curation of a unified dataset that consists of website screenshots, eye-gaze heatmap and website's l…
University of Science and Technology of China（中国科学技术大学）；Carnegie Mellon University（卡内基梅隆大学）
In recent years, head-mounted near-eye display devices have become the key hardware foundation for virtual reality and augmented reality. Thus head-mounted gaze tracking technology has received attention as an essential part of human-computer interaction. However, unavoidable slippage of head-mounted devices (HMD) often results higher gaze tracking errors and hinders the practical usage of HMD. To tackle this problem, we propose a slippage-robust gaze tracking for near-eye display method based on the aspheric eyeball model and accurately compute the eyeball optical axis and rotation center. We…
Conducting collaborative tasks, e.g., multi-user game, in virtual reality (VR) could enable us to explore more immersive and effective experience. However, for current VR systems, users cannot communicate properly with each other via their gaze points, and this would interfere with users' mutual understanding of the intention. In this study, we aimed to find the optimal eye tracking data visualization , which minimized the cognitive interference and improved the understanding of the visual attention and intention between users. We designed three different eye tracking data visualizations: gaze…
In this paper, we propose an approach to track the progression of eye-gaze while reading a block of text on computer screen. The proposed approach will help to accurately quantify reading, e.g., identifying the lines of text that were read/skipped and estimating the time spent on each line, based on commercially available inexpensive eye-tracking devices. The proposed approach is based on a novel Slip Kalman filter that is custom designed to track the progression of reading. The performance of the proposed method is demonstrated using 25 pages eye-tracking data collected using a commercial des…
Despite a revolution in the pervasiveness of video cameras in our daily lives, one of the most meaningful forms of nonverbal affective communication, interpersonal eye gaze, i.e. eye gaze relative to a conversation partner, is not available from common video. We introduce the Interpersonal-Calibrating Eye-gaze Encoder (ICE), which automatically extracts interpersonal gaze from video recordings without specialized hardware and without prior knowledge of participant locations. Leveraging the intuition that individuals spend a large portion of a conversation looking at each other enables the ICE …
Pupil Labs UG（Pupil Labs公司）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
Commercial head-mounted eye trackers provide useful features to customers in industry and research but are expensive and rely on closed source hardware and software. This limits the application areas and use of mobile eye tracking to expert users and inhibits user-driven development, customisation, and extension. In this paper we present Pupil -- an accessible, affordable, and extensible open source platform for mobile eye tracking and gaze-based interaction. Pupil comprises 1) a light-weight headset with high-resolution cameras, 2) an open source software framework for mobile eye tracking, as…
In this paper we consider the problem of tracking the progression of reading through eye-gaze measurements. Such an algorithm is novel and will ultimately help to develop a method of analyzing eye-gaze data which had been collected during reading activity in order to uncover crucial information regarding the individual's interest level and quality of experience while reading a passage of text or book. Additionally, such an approach will serve as a "visual signature" - a method of verifying if an individual has indeed given adequate attention to critical text-based information. Further, an accu…
In this paper, we firstly present what is Interactive Evolutionary Computation (IEC) and rapidly how we have combined this artificial intelligence technique with an eye-tracker for visual optimization. Next, in order to correctly parameterize our application, we present results from applying data mining techniques on gaze information coming from experiments conducted on about 80 human individuals.
Eye tracking tools are used in software engineering research to study various software development activities. However, a major limitation of these tools is their inability to track gaze data for activities that involve source code editing. We present a novel solution to support eye tracking experiments for tasks involving source code edits as an extension of the iTrace community infrastructure. We introduce the iTrace-Atom plugin and gazel -- a Python data processing pipeline that maps gaze information to changing source code elements and provides researchers with a way to query this dynamic …
Eye gaze controlled interfaces allow us to directly manipulate a graphical user interface just by looking at it. This technology has great potential in military aviation, in particular, operating different displays in situations where pilots hands are occupied with flying the aircraft. This paper reports studies on analyzing accuracy of eye gaze controlled interface inside aircraft undertaking representative flying missions. We reported that pilots can undertake representative pointing and selection tasks at less than 2 secs on average. Further, we evaluated the accuracy of eye gaze tracking g…
Indian Institute of Science（印度科学学院）；Indian Institute of Technology, Madras（马德拉斯印度理工学院）；Indian Institute of Information Technology, Kalyani（卡利亚尼印度信息技术学院）
Background: People with severe speech and motor impairment (SSMI) often uses a technique called eye pointing to communicate with outside world. One of their parents, caretakers or teachers hold a printed board in front of them and by analyzing their eye gaze manually, their intentions are interpreted. This technique is often error prone and time consuming and depends on a single caretaker. Objective: We aimed to automate the eye tracking process electronically by using commercially available tablet, computer or laptop and without requiring any dedicated hardware for eye gaze tracking. The eye …
Deep Learning-Based Visual Fatigue Detection Using Eye Gaze Patterns in VR
Dept. of Computer Science Clarkson University Potsdam, New York, USA（计算机科学系 克拉克逊大学）；David D. Reh School of Business Clarkson University Potsdam, New York, USA（戴维·D·雷商学院 克拉克逊大学）
Massachusetts Institute of Technology (MIT)（麻省理工学院）；Chalmers University of Technology（查尔姆斯理工大学）；SAFER
Accurate, robust, inexpensive gaze tracking in the car can help keep a driver safe by facilitating the more effective study of how to improve (1) vehicle interfaces and (2) the design of future Advanced Driver Assistance Systems. In this paper, we estimate head pose and eye pose from monocular video using methods developed extensively in prior work and ask two new interesting questions. First, how much better can we classify driver gaze using head and eye pose versus just using head pose? Second, are there individual-specific gaze strategies that strongly correlate with how much gaze classific…
Peking University（北京大学）；Southern University of Science and Technology（南方科技大学）
Hearing-impaired listeners usually have troubles attending target talker in multi-talker scenes, even with hearing aids (HAs). The problem can be solved with eye-gaze steering HAs, which requires listeners eye-gazing on the target. In a situation where head rotates, eye-gaze is subject to both behaviors of saccade and head rotation. However, existing methods of eye-gaze estimation did not work reliably, since the listener's strategy of eye-gaze varies and measurements of the two behaviors were not properly combined. Besides, existing methods were based on hand-craft features, which could overl…
Analyzing the gaze accuracy characteristics of an eye tracker is a critical task as its gaze data is frequently affected by non-ideal operating conditions in various consumer eye tracking applications. In this study, gaze error patterns produced by a commercial eye tracking device were studied with the help of machine learning algorithms, such as classifiers and regression models. Gaze data were collected from a group of participants under multiple conditions that commonly affect eye trackers operating on desktop and handheld platforms. These conditions (referred here as error sources) include…
University of California Berkeley（加州大学伯克利分校）；Stanford University（斯坦福大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
The cameras in modern gaze-tracking systems suffer from fundamental bandwidth and power limitations, constraining data acquisition speed to 300 Hz realistically. This obstructs the use of mobile eye trackers to perform, e.g., low latency predictive rendering, or to study quick and subtle eye motions like microsaccades using head-mounted devices in the wild. Here, we propose a hybrid frame-event-based near-eye gaze tracking system offering update rates beyond 10,000 Hz with an accuracy that matches that of high-end desktop-mounted commercial trackers when evaluated in the same conditions. Our s…
