Virtual reality (VR) systems can enable convenient hand-based interactions across diverse work scenarios. However, mid-air gestures lack tactile feedback and a physical reference surface to support the hand. This absence of haptic grounding can cause significant challenges in achieving precise and efficient touch interactions. This paper investigates the effect of different types of hand-grounded haptic feedback on the touch performance of VR tasks that demand high precision, such as selecting, tracing, and sketching. We compared three levels of haptic feedback: 1) No Haptic Feedback, where only visual feedback was provided; 2) Tactile Feedback, where users received vibrotactile and pressure feedback upon touching a virtual surface; 3) Physical Surface, where users interacted with a portable and tangible surface.
Our study found that portable physical surfaces enabled the best selection precision, tracing efficiency, and sketch quality. Furthermore, participants showed increased bimanual hand utilization when engaging with a physical surface during tasks. These observed behaviors corresponded to participants' preference for interacting with physical surfaces, attributed to a better sense of confidence and control.
We propose a new technique to help users to explore geometric shapes without vision. This technique is based on a guidance using directional cues with a pin array. This is an alternative to the usual technique that consists of raising the pins corresponding to dark pixels around the cursor. In this paper we compare the exploration of geometric shapes with our new technique in unimanual and bimanual conditions. The users made fewer errors in unimanual condition than in bimanual condition. However they did not explore the shapes more quickly and there was no difference in confidence in their ans…
In this paper, we propose a new interface for virtual reality headset: a touchpad in front of the headset. To demonstrate the feasibility of the front touch interface, we built a prototype device, explored VR UI design space expansion, and performed various user studies. We started with preliminary tests to see how intuitively and accurately people can interact with the front touchpad. Then, we further experimented various user interfaces such as a binary selection, a typical menu layout, and a keyboard. Two-Finger and Drag-n-Tap were also explored to find the appropriate selection technique. …
This is the preprint version of our paper on Personal and Ubiquitous Computing. There is an increasing interest in creating pervasive games based on emerging interaction technologies. In order to develop touch-less, interactive and augmented reality games on vision-based wearable device, a touch-less motion interaction technology is designed and evaluated in this work. Users interact with the augmented reality games with dynamic hands/feet gestures in front of the camera, which triggers the interaction event to interact with the virtual object in the scene. Three primitive augmented reality ga…
HandPad: A Bimanual Hand Interface for Fluid Window Interactions in VR
Department of Computer Science, University of Virginia（弗吉尼亚大学计算机科学系）；Department of Computer Science and Engineering, Ulsan National Institute of Science and Technology（蔚山科学技术院计算机科学与工程系）
In this paper, we describe the results of replicating one of our studies from two years ago which compares two techniques for transferring content across multiple screens in VR. Results from the previous study have shown that a combined gaze and touch input can outperform a bimanual touch-only input in terms of task completion time, simulator sickness, task load and usability. Except for the simulator sickness, these findings could be validated by the replication. The difference with regards to simulator sickness and variations in absolute scores of the other measures could be explained by a d…
This study aims to better understand the emotional and physiological correlates of being caressed in VR depending on the type of multisensory feedback provided and the animate or inanimate nature of the virtual representation that touches an embodied virtual body. We evaluated how pleasure, arousal, embodiment, and the illusion of being touched in VR were influenced by the inclusion of only visual feedback compared to visuotactile stimulation conditions, where participants, in addition to seeing an avatar or feather caressing their virtual bodies, also perceived congruent mid-air ultrasonic ta…
Haptic feedback has become crucial to enhance the user experiences in Virtual Reality (VR). This justifies the sudden burst of novel haptic solutions proposed these past years in the HCI community. This article is a survey of Virtual Reality interactions, relying on haptic devices. We propose two dimensions to describe and compare the current haptic solutions: their degree of physicality, as well as their degree of actuation. We depict a compromise between the user and the designer, highlighting how the range of required or proposed stimulation in VR is opposed to the haptic interfaces flexibi…
Virtual Reality (VR) has the potential to transform knowledge work. One advantage of VR knowledge work is that it allows extending 2D displays into the third dimension, enabling new operations, such as selecting overlapping objects or displaying additional layers of information. On the other hand, mobile knowledge workers often work on established mobile devices, such as tablets, limiting interaction with those devices to a small input space. This challenge of a constrained input space is intensified in situations when VR knowledge work is situated in cramped environments, such as airplanes an…
Shared Boundary Interfaces: can one fit all? A controlled study on virtual reality vs touch-screen interfaces on persons with Neurodevelopmental Disorders
An increasing number of consumer-oriented head-mounted displays (HMD) for augmented and virtual reality (AR/VR) are capable of finger and hand tracking. We report on the accuracy of off-the-shelf VR and AR HMDs when used for touch-based tasks such as pointing or drawing. Specifically, we report on the finger tracking accuracy of the VR head-mounted displays Oculus Quest, Vive Pro and the Leap Motion controller, when attached to a VR HMD, as well as the finger tracking accuracy of the AR head-mounted displays Microsoft HoloLens 2 and Magic Leap. We present the results of two experiments in whic…
Noninvasive brain-computer interface (BCI) is widely used to recognize users' intentions. Especially, BCI related to tactile and sensation decoding could provide various effects on many industrial fields such as manufacturing advanced touch displays, controlling robotic devices, and more immersive virtual reality or augmented reality. In this paper, we introduce haptic and sensory perception-based BCI systems called neurohaptics. It is a preliminary study for a variety of scenarios using actual touch and touch imagery paradigms. We designed a novel experimental environment and a device that co…
We study the performance and user experience of two popular mainstream text entry devices, desktop keyboards and touchscreen keyboards, for use in Virtual Reality (VR) applications. We discuss the limitations arising from limited visual feedback, and examine the efficiency of different strategies of use. We analyze a total of 24 hours of typing data in VR from 24 participants and find that novice users are able to retain about 60% of their typing speed on a desktop keyboard and about 40-45\% of their typing speed on a touchscreen keyboard. We also find no significant learning effects, indicati…
Since the launch of Google Glass in 2014, smart glasses have mainly been designed to support micro-interactions. The ultimate goal for them to become an augmented reality interface has not yet been attained due to an encumbrance of controls. Augmented reality involves superimposing interactive computer graphics images onto physical objects in the real world. This survey reviews current research issues in the area of human computer interaction for smart glasses. The survey first studies the smart glasses available in the market and afterwards investigates the interaction methods proposed in the…
We contribute interaction techniques for augmenting mixed reality (MR) visualizations with smartphone proxies. By combining head-mounted displays (HMDs) with mobile touchscreens, we can augment low-resolution holographic 3D charts with precise touch input, haptics feedback, high-resolution 2D graphics, and physical manipulation. Our approach aims to complement both MR and physical visualizations. Most current MR visualizations suffer from unreliable tracking, low visual resolution, and imprecise input. Data physicalizations on the other hand, although allowing for natural physical manipulation…
Virtual Reality (VR) headsets can open opportunities for users to accomplish complex tasks on large virtual displays, using compact setups. However, interacting with large virtual displays using existing interaction techniques might cause fatigue, especially for precise manipulations, due to the lack of physical surfaces. We designed VXSlate, an interaction technique that uses a large virtual display, as an expansion of a tablet. VXSlate combines a user's headmovement, as tracked by the VR headset, and touch interaction on the tablet. The user's headmovement position both a virtual representat…
The information provided to a person's visual system by extended reality (XR) displays is not a veridical match to the information provided by the real world. Due in part to graphical limitations in XR head-mounted displays (HMDs), which vary by device, our perception of space may be altered. However, we do not yet know which properties of virtual objects rendered by HMDs -- particularly augmented reality displays -- influence our ability to understand space. In the current research, we evaluate how immersive graphics affect spatial perception across three unique XR displays: virtual reality (…
Virtual Reality enables the exploration of large information spaces. In physically constrained spaces such as airplanes or buses, controller-based or mid-air interaction in mobile Virtual Reality can be challenging. Instead, the input space on and above touch-screen enabled devices such as smartphones or tablets could be employed for Virtual Reality interaction in those spaces. In this context, we compared an above surface interaction technique with traditional 2D on-surface input for navigating large planar information spaces such as maps in a controlled user study (n = 20). We find that our …
Research in user interaction with mixed reality environments using multiple displays has become increasingly relevant with the prevalence of mobile devices in everyday life and increased commoditization of large display area technologies using projectors or large displays. Previous work often combines touch-based input with other approaches, such as gesture-based input, to expand the possible interaction space or deal with limitations of other two-dimensional input methods. In contrast to previous methods, we examine the possibilities when the control-display (C-D) ratio is significantly small…
We present PhyShare, a new haptic user interface based on actuated robots. Virtual reality has recently been gaining wide adoption, and an effective haptic feedback in these scenarios can strongly support user's sensory in bridging virtual and physical world. Since participants do not directly observe these robotic proxies, we investigate the multiple mappings between physical robots and virtual proxies that can utilize the resources needed to provide a well rounded VR experience. PhyShare bots can act either as directly touchable objects or invisible carriers of physical objects, depending on…
Augmented Reality (AR) and Mobile Augmented Reality (MAR) applications have gained much research and industry attention these days. The mobile nature of MAR applications limits users' interaction capabilities such as inputs, and haptic feedbacks. This survey reviews current research issues in the area of human computer interaction for MAR and haptic devices. The survey first presents human sensing capabilities and their applicability in AR applications. We classify haptic devices into two groups according to the triggered sense: cutaneous/tactile: touch, active surfaces, and mid-air, kinesthet…
Virtual Reality (VR) has the potential to support mobile knowledge workers by complementing traditional input devices with a large three-dimensional output space and spatial input. Previous research on supporting VR knowledge work explored domains such as text entry using physical keyboards and spreadsheet interaction using combined pen and touch input. Inspired by such work, this paper probes the VR design space for authoring presentations in mobile settings. We propose PoVRPoint -- a set of tools coupling pen- and touch-based editing of presentations on mobile devices, such as tablets, with …
We present a prototype demonstrator that integrates three technologies, mixed reality head-mounted displays, wearable bio-sensors, and mid-air haptic projectors to deliver an interactive tactile experience with a bio-hologram. Users of this prototype are able to see, touch and feel a hologram of a heart that is beating at the same rhythm as their own. The demo uses an Ultrahaptics device, a Magic Leap One Mixed Reality headset, and an Apple Watch that measures the wearer's heart rate, all synchronized and networked together such that updates from the wristband dynamically change the haptic fee…
This workshop paper presents two proxy objects for high fidelity interaction in virtual reality (VR): a paper map and a smartphone. We showcase how our virtual paper map can increase interactivity and orientation, while our virtual smartphone extends the use of a proxy object, as it allows for actual touch input on a real phone leading to an almost infinite set of possible (inter-)actions (e.g. snapping pictures in the virtual world). Observations showed that participants were very precise in holding and interacting with both the paper map and the smartphone even though they did not see their …
As the capability and complexity of UAVs continue to increase, the human-robot interface community has a responsibility to design better ways of specifying the complex 3D flight paths necessary for instructing them. Immersive interfaces, such as those afforded by virtual reality (VR), have several unique traits which may improve the user's ability to perceive and specify 3D information. These traits include stereoscopic depth cues which induce a sense of physical space as well as six degrees of freedom (DoF) natural head-pose and gesture interactions. This work introduces an open-source platfo…
In this paper we introduce novel Virtual Reality (VR) and Augmented Reality (AR) treatments to improve the psychological well being of patients in palliative care, based on interviews with a clinical psychologist who has successfully implemented VR assisted interventions on palliative care patients in the Hong Kong hospital system. Our VR and AR assisted interventions are adaptations of traditional palliative care therapies which simultaneously facilitate patients communication with family and friends while isolated in hospital due to physical weakness and COVID-19 related restrictions. The fi…
Chinese Academy of Science（中国科学院）；Royal Institute of Technology (KTH)（瑞典皇家理工学院）
This is the preprint version of our paper on IEEE Virtual Reality Conference 2015. A touch-less interaction technology on vision based wearable device is designed and evaluated. Users interact with the application with dynamic hands/feet gestures in front of the camera. Several proof-of-concept prototypes with eleven dynamic gestures are developed based on the touch-less interaction. At last, a comparing user study evaluation is proposed to demonstrate the usability of the touch-less approach, as well as the impact on user's emotion, running on a wearable framework or Google Glass.
Virtual reality (VR) is an important new technology that is fun-damentally changing the way people experience entertainment and education content. Due to the fact that most currently available VR products are one size fits all, the accessibility of the content design and user interface design, even for healthy children is not well understood. It requires more research to ensure that children can have equally good user compared to adults in VR. In our study, we seek to explore accessibility of locomotion in VR between healthy adults and minors along both objective and subjective dimensions. We …
This paper proposes the concept of "live-action virtual reality games" as a new genre of digital games based on an innovative combination of live-action, mixed-reality, context-awareness, and interaction paradigms that comprise tangible objects, context-aware input devices, and embedded/embodied interactions. Live-action virtual reality games are "live-action games" because a player physically acts out (using his/her real body and senses) his/her "avatar" (his/her virtual representation) in the game stage, which is the mixed-reality environment where the game happens. The game stage is a kind …
Navigating spaces is an embodied experience. Examples can vary from rescue workers trying to save people from natural disasters; a tourist finding their way to the nearest coffee shop, or a gamer solving a maze. Virtual reality allows these experiences to be simulated in a controlled virtual environment. However, virtual reality users remain anchored in the real world and the conventions by which the virtual environment is deployed influence user performance. There is currently a need to evaluate the degree of influence imposed by extrinsic factors and virtual reality hardware on its users. Tr…
This paper presents a design space of interaction techniques to engage with visualizations that are printed on paper and augmented through Augmented Reality. Paper sheets are widely used to deploy visualizations and provide a rich set of tangible affordances for interactions, such as touch, folding, tilting, or stacking. At the same time, augmented reality can dynamically update visualization content to provide commands such as pan, zoom, filter, or detail on demand. This paper is the first to provide a structured approach to mapping possible actions with the paper to interaction commands. Thi…
The exciting new technology known as mid-air haptics has been adopted by several industries including Automotive and Entertainment, however it has yet to emerge in simulated pilot training or in real-life flight decks. Full-flight simulators are expensive to manufacture, maintain and operate. Not only that, each simulator is limited to one aircraft type, which is inefficient for the majority of airlines that have several in service. With the growing trend in touchscreen instrumentation, cockpit displays require the pilot's attention to be drawn away from their view out of the window. But by us…
Ageing, Digital Literacy, and Interaction Modality in Immer-sive Virtual Reality: Psychomotor Performance, Cognitive Flexibility, and Their Processing-Speed Association
National and Kapodistrian University of Athens（雅典国立卡波迪斯特里阿斯大学）；The American College of Greece（希腊美国学院）；University of Edinburgh（爱丁堡大学）
MetaSpace II (MS2) is a social Virtual Reality (VR) system where multiple users can not only see and hear but also interact with each other, grasp and manipulate objects, walk around in space, and get tactile feedback. MS2 allows walking in physical space by tracking each user's skeleton in real-time and allows users to feel by employing passive haptics i.e., when users touch or manipulate an object in the virtual world, they simultaneously also touch or manipulate a corresponding object in the physical world. To enable these elements in VR, MS2 creates a correspondence in spatial layout and o…
Physical keyboards are common peripherals for personal computers and are efficient standard text entry devices. Recent research has investigated how physical keyboards can be used in immersive head-mounted display-based Virtual Reality (VR). So far, the physical layout of keyboards has typically been transplanted into VR for replicating typing experiences in a standard desktop environment. In this paper, we explore how to fully leverage the immersiveness of VR to change the input and output characteristics of physical keyboard interaction within a VR environment. This allows individual physica…
The state of the art review broadly oversees the use of novel research utilized in the creation of virtual environments applied in interactive art experiences, with a specific focus on the application of procedural animation in spatially augmented reality (SAR) exhibitions. These art exhibitions frequently combine sensory displays that appeal, replace, and augment the visual, auditory and touch or haptic senses. We analyze and break down art-technology related innovations in the last three years, and thoroughly identify the most recent and vibrant applications of interactive art experiences in…
This research investigates whether the interaction methods of XR headsets can be improved by using haptic feedback. As a first and most common technique, indirect interactions are considered. Indirect interactions correspond to manipulations of virtual objects from a virtual distance using pre-defined hand gestures. As a second interaction technique, direct interaction (namely DIM) has been implemented where the user manipulates objects by virtually touching these with their hands. A third interaction method extends the previous one with haptic feedback (namely HEDIM). These 3 methods are comp…
Virtual Reality (VR) can enhance the display and interaction of mobile knowledge work and in particular, spreadsheet applications. While spreadsheets are widely used yet are challenging to interact with, especially on mobile devices, using them in VR has not been explored in depth. A special uniqueness of the domain is the contrast between the immersive and large display space afforded by VR, contrasted by the very limited interaction space that may be afforded for the information worker on the go, such as an airplane seat or a small work-space. To close this gap, we present a tool-set for enh…
