Eye Tracking (ET) can help to understand visual attention and cognitive processes in interactive environments. In attention tasks, distinguishing between relevant target objects and distractors is crucial for effective performance, yet the underlying gaze patterns that drive successful task completion remain incompletely understood. Traditional gaze analyses lack comprehensive insights into the temporal dynamics of attention allocation and the relationship between gaze behavior and task performance. When applied to complex visual search scenarios, current gaze analysis methods face several limitations, including the isolation of measurements, visual stability, search efficiency, and the decision-making processes involved in these scenarios. This paper proposes an analysis tool that considers time series for eye tracking data from task performance and also gaze measures (fixations, saccades and smooth pursuit); temporal pattern analysis that reveals how attention evolves throughout task performance; object-click sequence tracking that directly links visual attention to user actions; and performance metrics that quantify both accuracy and efficiency. This tool provides comprehensive visualization techniques that make complex patterns of stimuli and gaze connections interpretable.
Technical progress in hardware and software enables us to record gaze data in everyday situations and over long time spans. Among a multitude of research opportunities, this technology enables visualization researchers to catch a glimpse behind performance measures and into the perceptual and cognitive processes of people using visualization techniques. The majority of eye tracking studies performed for visualization research is limited to the analysis of gaze distributions and aggregated statistics, thus only covering a small portion of insights that can be derived from gaze data. We argue th…
Eye movements hold information about human perception, intention, and cognitive state. We propose a novel eye movement simulator that i) probabilistically simulates saccade movements as gamma distributions considering different peak velocities and ii) models smooth pursuit onsets with the sigmoid function. Additionally, it is capable of producing velocity and two-dimensional gaze sequences for static and dynamic scenes using saliency maps or real fixation targets. Our approach is also capable of simulating any sampling rate, even with uctuations. The simulation is evaluated against publicly av…
Cognitive biases are systematic errors in judgment. Researchers in data visualizations have explored whether cognitive biases transfer to decision-making tasks with interactive data visualizations. At the same time, cognitive scientists have reinterpreted cognitive biases as the product of resource-rational strategies under finite time and computational costs. In this paper, we argue for the integration of resource-rational analysis through constrained Bayesian cognitive modeling to understand cognitive biases in data visualizations. The benefit would be a more realistic "bounded rationality" …
Eyes Can't Always Tell: Fusing Eye Tracking and User Priors for User Modeling under AI Advice Conditions
National Institute of Informatics（国立情报学研究所）；Yale School of Medicine（耶鲁大学医学院）；University of Amsterdam（阿姆斯特丹大学）；Centrum Wiskunde & Informatica（数学与计算机科学中心）；Utrecht University（乌得勒支大学）
We examine if EEG-based cognitive load (CL) estimation is generalizable across the character, spatial pattern, bar graph and pie chart-based visualizations for the nback~task. CL is estimated via two recent approaches: (a) Deep convolutional neural network, and (b) Proximal support vector machines. Experiments reveal that CL estimation suffers across visualizations motivating the need for effective machine learning techniques to benchmark visual interface usability for a given analytic task.
Comparing mobile eye tracking data from multiple participants without information about areas of interest (AOIs) is challenging because of individual timing and coordinate systems. We present a technique, the gaze spiral, that visualizes individual recordings based on image content of the stimulus. The spiral layout of the slitscan visualization is used to create a compact representation of scanpaths. The visualization provides an overview of multiple recordings even for long time spans and helps identify and annotate recurring patterns within recordings. The gaze spirals can also serve as gly…
Augmented reality (AR) technology provides means for embedding visualization in a real-world context. Such techniques allow situated analyses of live data in their spatial domain. However, as existing techniques have to be adapted for this context and new approaches will be developed, the evaluation thereof poses new challenges for researchers. Apart from established performance measures, eye tracking has proven to be a valuable means to assess visualizations qualitatively and quantitatively. We discuss the challenges and opportunities of eye tracking for the evaluation of situated visualizati…
Investigations into using visualization to improve Bayesian reasoning and advance risk communication have produced mixed results, suggesting that cognitive ability might affect how users perform with different presentation formats. Our work examines the cognitive load elicited when solving Bayesian problems using icon arrays, text, and a juxtaposition of text and icon arrays. We used a three-pronged approach to capture a nuanced picture of cognitive demand and measure differences in working memory capacity, performance under divided attention using a dual-task paradigm, and subjective ratings …
We present an algorithmic and visual grouping of participants and eye-tracking metrics derived from recorded eye-tracking data. Our method utilizes two well-established visualization concepts. First, parallel coordinates are used to provide an overview of the used metrics, their interactions, and similarities, which helps select suitable metrics that describe characteristics of the eye-tracking data. Furthermore, parallel coordinates plots enable an analyst to test the effects of creating a combination of a subset of metrics resulting in a newly derived eye-tracking metric. Second, a similarit…
Eye movement patterns reflect human latent internal cognitive activities. We aim to discover eye movement patterns during face recognition under different cognitions of information concealing. These cognitions include the degrees of face familiarity and deception or not, namely telling the truth when observing familiar and unfamiliar faces, and deceiving in front of familiar faces. We apply Hidden Markov models with Gaussian emission to generalize regions and trajectories of eye fixation points under the above three conditions. Our results show that both eye movement patterns and eye gaze regi…
Misinformation is an important topic in the Information Retrieval (IR) context and has implications for both system-centered and user-centered IR. While it has been established that the performance in discerning misinformation is affected by a person's cognitive load, the variation in cognitive load in judging the veracity of news is less understood. To understand the variation in cognitive load imposed by reading news headlines related to COVID-19 claims, within the context of a fact-checking system, we conducted a within-subject, lab-based, quasi-experiment (N=40) with eye-tracking. Our resu…
This paper proposes a set of techniques to investigate eye gaze and fixation patterns while users interact with electronic user interfaces. In particular, two case studies are presented - one on analysing eye gaze while interacting with deceptive materials in web pages and another on analysing graphs in standard computer monitor and virtual reality displays. We analysed spatial and temporal distributions of eye gaze fixations and sequence of eye gaze movements. We used this information to propose new design guidelines to avoid deceptive materials in web and user-friendly representation of data…
Conducting collaborative tasks, e.g., multi-user game, in virtual reality (VR) could enable us to explore more immersive and effective experience. However, for current VR systems, users cannot communicate properly with each other via their gaze points, and this would interfere with users' mutual understanding of the intention. In this study, we aimed to find the optimal eye tracking data visualization , which minimized the cognitive interference and improved the understanding of the visual attention and intention between users. We designed three different eye tracking data visualizations: gaze…
Cognitive load theory (CLT) provides us guiding principles in the design of learning materials. CLT differentiates three different kinds of cognitive load -- intrinsic, extraneous and germane load. Intrinsic load is related to the learning goal, extraneous load costs cognitive resources but does not contribute to learning. Germane load can foster learning. Objective methods, such as eye movement measures and EEG have been used measure the total cognitive load. Very few research studies, if any, have been completed to measure the three kinds of load separately with physiological methods in a co…
Gaze-based virtual keyboards provide an effective interface for text entry by eye movements. The efficiency and usability of these keyboards have traditionally been evaluated with conventional text entry performance measures such as words per minute, keystrokes per character, backspace usage, etc. However, in comparison to the traditional text entry approaches, gaze-based typing involves natural eye movements that are highly correlated with human brain cognition. Employing eye gaze as an input could lead to excessive mental demand, and in this work we argue the need to include cognitive load a…
Preattentive visual features such as hue or flickering can effectively draw attention to an object of interest -- for instance, an important feature in a scientific visualization. These features appear to pop out and can be recognized by our visual system, independently from the number of distractors. Most cues do not take advantage of the fact that most humans have two eyes. In cases where binocular vision is applied, it is almost exclusively used to convey depth by exposing stereo pairs. We present Deadeye, a novel preattentive visualization technique based on presenting different stimuli to…
CLERA: A Unified Model for Joint Cognitive Load and Eye Region Analysis in the Wild
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；University of California San Diego（加利福尼亚大学圣迭戈分校）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；Massachusetts Institute of Technology（麻省理工学院）
Usage of smartphones and tablets have been increasing rapidly with multi-touch interaction and powerful configurations. Performing tasks on mobile phones become more complex as people age, thereby increasing their cognitive workload. In this context, we conducted an eye tracking study with 50 participants between the age of 20 to 60 years and above, living in Bangalore, India. This paper focuses on visual nature of interaction with mobile user interfaces. The study aims to investigate how aging affects user experience on mobile phones while performing complex tasks, and estimate cognitive work…
This paper introduces the use of electroencephalography (EEG) and eye tracking in exploring customer experiences in service design. These tools are expected to allow designers to generate customer journeys from empirical data leading to new visualization methods and therefore improvements in service design deliverables.
Understanding your audience is foundational to creating high impact visualization designs. However, individual differences and cognitive abilities also influence interactions with information visualization. Differing user needs and abilities suggest that an individual's background could influence cognitive performance and interactions with visuals in a systematic way. This study builds on current research in domain-specific visualization and cognition to address if domain and spatial visualization ability combine to affect performance on information visualization tasks. We measure spatial visu…
There is growing evidence that cognitive processes may have fractal structures as a signature of complexity. It is an an ongoing topic of research to study the class of complexity and how it may differ as a function of cognitive variables. Here, we explore the eye movement trajectories generated during reading different Persian texts. Features of eye movement trajectories were recorded during reading Persian texts using an eye tracker. We show that fixation durations, as the main components of eye movements reflecting cognitive processing, exhibits multifractal behavior. This indicates that mu…
Deep Learning in Mild Cognitive Impairment Diagnosis using Eye Movements and Image Content in Visual Memory Tasks
Institute for Systems and Robotics（系统与机器人研究所）；Laboratory for Robotics and Engineering Systems（机器人与工程系统实验室）；Instituto Superior Técnico（技术高等学院）；Universidade de Lisboa（里斯本大学）；Department of Psychology（心理学系）；University of Chicago（芝加哥大学）；Institute for Mind and Biology（心灵与生物学研究所）；Sapienza Università di Roma（罗马萨皮恩扎大学）
University of Tübingen（图宾根大学）；Mahidol University（玛希隆大学）；University of Bremen（不来梅大学）；Ramathibodi Hospital（拉玛提波迪医院）
During arthroscopic surgeries, surgeons are faced with challenges like cognitive re-projection of the 2D screen output into the 3D operating site or navigation through highly similar tissue. Training of these cognitive processes takes much time and effort for young surgeons, but is necessary and crucial for their education. In this study we want to show how to recognize states of confusion of young surgeons during an arthroscopic surgery, by looking at their eye and head movements and feeding them to a machine learning model. With an accuracy of over 94\% and detection speed of 0.039 seconds, …
One of the major challenges for evaluating the effectiveness of data visualizations and visual analytics tools arises from the fact that different users may be using these tools for different tasks. In this paper, we present a simple example of how different tasks lead to different patterns of attention to the same underlying data visualizations. We argue that the general approach used in this experiment could be applied systematically to task and feature taxonomies that have been developed by visualization researchers. Using eye tracking to study the impact of common tasks on how humans atten…
