This research investigates whether the interaction methods of XR headsets can be improved by using haptic feedback. As a first and most common technique, indirect interactions are considered. Indirect interactions correspond to manipulations of virtual objects from a virtual distance using pre-defined hand gestures. As a second interaction technique, direct interaction (namely DIM) has been implemented where the user manipulates objects by virtually touching these with their hands. A third interaction method extends the previous one with haptic feedback (namely HEDIM). These 3 methods are compared with each other based on objective and subjective user tests, also taking into account financial considerations. This research concludes that the DIM improves upon the standard indirect method. Additionally, it has been observed that haptic feedback could enhance the DIM in specific situations. Nevertheless, when considering the current financial cost, our subjects were not convinced of the small improvements haptic feedback brings.
Augmented Reality (AR) and Mobile Augmented Reality (MAR) applications have gained much research and industry attention these days. The mobile nature of MAR applications limits users' interaction capabilities such as inputs, and haptic feedbacks. This survey reviews current research issues in the area of human computer interaction for MAR and haptic devices. The survey first presents human sensing capabilities and their applicability in AR applications. We classify haptic devices into two groups according to the triggered sense: cutaneous/tactile: touch, active surfaces, and mid-air, kinesthet…
Haptic feedback has become crucial to enhance the user experiences in Virtual Reality (VR). This justifies the sudden burst of novel haptic solutions proposed these past years in the HCI community. This article is a survey of Virtual Reality interactions, relying on haptic devices. We propose two dimensions to describe and compare the current haptic solutions: their degree of physicality, as well as their degree of actuation. We depict a compromise between the user and the designer, highlighting how the range of required or proposed stimulation in VR is opposed to the haptic interfaces flexibi…
We propose a new approach for interaction in Virtual Reality (VR) using mobile robots as proxies for haptic feedback. This approach allows VR users to have the experience of sharing and manipulating tangible physical objects with remote collaborators. Because participants do not directly observe the robotic proxies, the mapping between them and the virtual objects is not required to be direct. In this paper, we describe our implementation, various scenarios for interaction, and a preliminary user study.
We present a prototype demonstrator that integrates three technologies, mixed reality head-mounted displays, wearable bio-sensors, and mid-air haptic projectors to deliver an interactive tactile experience with a bio-hologram. Users of this prototype are able to see, touch and feel a hologram of a heart that is beating at the same rhythm as their own. The demo uses an Ultrahaptics device, a Magic Leap One Mixed Reality headset, and an Apple Watch that measures the wearer's heart rate, all synchronized and networked together such that updates from the wristband dynamically change the haptic fee…
In this paper, we present a low-cost virtual reality (VR) collaborative system equipped with a haptic feedback sensation system. This system is composed of a Kinect sensor for bodies and gestures detection, a microcontroller and vibrators to simulate outside interactions, and smartphone powered cardboard, all of this are put into a network implemented with Unity 3D game engine. CCS CONCEPTS $\bullet$ Interaction paradigms $\rightarrow$ Virtual reality; Collaborative interaction; $\bullet$ Hardware $\rightarrow$ Sensors and actuators; Wireless devices; KEYWORDS collaborative virtual reality, ha…
Stanford University（斯坦福大学）；Kadir Has University（卡迪尔哈斯大学）
Relocation of haptic feedback from the fingertips to the wrist has been considered as a way to enable haptic interaction with mixed reality virtual environments while leaving the fingers free for other tasks. We present a pair of wrist-worn tactile haptic devices and a virtual environment to study how various mappings between fingers and tactors affect task performance. The haptic feedback rendered to the wrist reflects the interaction forces occurring between a virtual object and virtual avatars controlled by the index finger and thumb. We performed a user study comparing four different finge…
Physical rehabilitation is essential to recovery from joint replacement operations. As a representation, total knee arthroplasty (TKA) requires patients to conduct intensive physical exercises to regain the knee's range of motion and muscle strength. However, current joint replacement physical rehabilitation methods rely highly on therapists for supervision, and existing computer-assisted systems lack consideration for enabling self-monitoring, making at-home physical rehabilitation difficult. In this paper, we investigated design recommendations that would enable self-monitored rehabilitation…
Haptic feedback in virtual reality (VR) allows users to perceive the physical properties of virtual objects (e.g., their weight and motion patterns). However, the lack of haptic sensations deteriorates users' immersion and overall experience. In this work, we designed and implemented a low-cost hardware prototype with liquid metal, VibroWeight, which can work in complementarity with commercial VR handheld controllers. VibroWeight is characterized by bimodal feedback cues in VR, driven by adaptive absolute mass (weights) and gravity shift. To our knowledge, liquid metal is used in a VR haptic d…
In mixed reality, real objects can be used to interact with virtual objects. However, unlike in the real world, real objects do not encounter any opposite reaction force when pushing against virtual objects. The lack of reaction force during manipulation prevents users from perceiving the mass of virtual objects. Although this could be addressed by equipping real objects with force-feedback devices, such a solution remains complex and impractical.In this work, we present a technique to produce an illusion of mass without any active force-feedback mechanism. This is achieved by simulating the e…
Grasping is an incredible ability of animals using their arms and limbs in their daily life. The human hand is an especially astonishing multi-fingered tool for precise grasping, which helped humans to develop the modern world. The implementation of the human grasp to virtual reality and telerobotics is always interesting and challenging at the same time. In this work, authors surveyed, studied, and analyzed the human hand-grasping behavior for the possibilities of haptic grasping in the virtual and remote environment. This work is focused on the motion and force analysis of fingers in human h…
This paper presents I-nteract, a cyber-physical system that enables real-time interaction with real and virtual objects in a mixed augmented reality environment to design 3D models for additive manufacturing. The system has been developed using mixed reality technologies such as HoloLens, for augmenting visual feedback, and haptic gloves, for augmenting haptic force feedback. The efficacy of the system has been demonstrated by generating 3D model using a novel scanning method to 3D print a customized orthopedic cast for human arm, by estimating spring rates of compression springs, and by simul…
Skolkovo Institute of Science and Technology（斯科尔科沃科技学院）
We propose a novel concept of augmented reality (AR) human-drone interaction driven by RL-based swarm behavior to achieve intuitive and immersive control of a swarm formation of unmanned aerial vehicles. The DroneARchery system developed by us allows the user to quickly deploy a swarm of drones, generating flight paths simulating archery. The haptic interface LinkGlide delivers a tactile stimulus of the bowstring tension to the forearm to increase the precision of aiming. The swarm of released drones dynamically avoids collisions between each other, the drone following the user, and external o…
Mid-air interaction is one of the promising interaction modalities in virtual reality (VR) due to its merits in naturalness and intuitiveness, but the interaction suffers from the lack of haptic feedback as no force or vibrotactile feedback can be provided in mid-air. As a breakthrough to compensate for this insufficiency, the application of pseudo-haptic features which create the visuo-haptic illusion without actual physical haptic stimulus can be explored. Therefore, this study aimed to investigate the effect of four pseudo-haptic features: proximity feedback, protrusion, hit effect, and pen…
Evaluating the efficacy of haptic feedback, 360° treadmill-integrated Virtual Reality framework and longitudinal training on decision-making performance in a complex search-and-shoot simulation
We present PhyShare, a new haptic user interface based on actuated robots. Virtual reality has recently been gaining wide adoption, and an effective haptic feedback in these scenarios can strongly support user's sensory in bridging virtual and physical world. Since participants do not directly observe these robotic proxies, we investigate the multiple mappings between physical robots and virtual proxies that can utilize the resources needed to provide a well rounded VR experience. PhyShare bots can act either as directly touchable objects or invisible carriers of physical objects, depending on…
HapticGiant: A Novel Very Large Kinesthetic Haptic Interface with Hierarchical Force Control
Intelligent Sensor-Actuator-Systems Laboratory (ISAS), Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT)（智能传感器-执行器系统实验室（ISAS），人机学与机器人研究所，卡尔斯鲁厄理工学院（KIT））
While user's perception & performance are predominantly examined independently in virtual reality, the Action-Specific Perception (ASP) theory postulates that the performance of an individual on a task modulates this individual's spatial & time perception pertinent to the task's components & procedures. This paper examines the association between performance & perception & the potential effects that tactile feedback modalities could generate. This paper reports a user study (N=24), in which participants performed a Fitts's law target acquisition task by using three feedback modalities: visual,…
Presence in virtual reality (VR) is meaningful for remotely connecting with others and facilitating social interactions despite great distance while providing a sense of "being there." This work presents CardsVR, a two-person VR experience that allows remote participants to play a game of cards together. An entire deck of tracked cards are used to recreate the sense of playing cards in-person. Prior work in VR commonly provides passive haptic feedback either through a single object or through static objects in the environment. CardsVR is novel in providing passive haptic feedback through multi…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
We propose a concept of a novel interaction strategy for providing rich haptic feedback in Virtual Reality (VR), when each user's finger is connected to micro-quadrotor with a wire. Described technology represents the first flying wearable haptic interface. The solution potentially is able to deliver high resolution force feedback to each finger during fine motor interaction in VR. The tips of tethers are connected to the centers of quadcopters under their bottom. Therefore, flight stability is increasing and the interaction forces are becoming stronger which allows to use smaller drones.
This chapter gives an overview of interaction techniques for mixed reality including augmented and virtual reality (AR/VR). Various modalities for input and output are discussed. Specifically, techniques for tangible and surface-based interaction, gesture-based, pen-based, gaze-based, keyboard and mouse-based, as well as haptic interaction are discussed. Furthermore, the combination of multiple modalities in multisensory and multimodal interaction, as well as interaction using multiple physical or virtual displays, are presented. Finally, interaction with intelligent virtual agents is consider…
Armstrong Atlantic State Univ.（阿姆斯特朗大西洋州立大学）；Ovidius University（奥维迪乌斯大学）
Surgeons must accomplish complex technical and intellectual tasks that can generate unexpected and serious challenges with little or no room for error. In the last decade, computer simulations have played an increasing role in surgical training, pre-operative planning, and biomedical research. Specifically, visuo-haptic simulations have been the focus of research to develop advanced e-Learning systems facilitating surgical training. The cost of haptic hardware was reduced through mass scale production and as haptics gained popularity in the gaming industry. Visuo-haptic simulations combine the…
Haptic feedback is essential to acquire immersive experience when interacting in virtual or augmented reality. Although the existing promising magnetic levitation (maglev) haptic system has advantages of none mechanical friction, its performance is limited by its navigation method, which mainly results from the challenge that it is difficult to obtain high precision, high frame rate and good stability with lightweight design at the same. In this study, we propose to perform the visual-inertial fusion navigation based on sequence-to-sequence learning for the maglev haptic interaction. Cascade L…
Towards Tangible Immersion for Cobot Programming-by-Demonstration: Visual, Tactile and Haptic Interfaces for Mixed-Reality Cobot Automation in Semiconductor Manufacturing
Intel Labs, Intelligent System Research, Human-Robot Collaboration（英特尔实验室、智能系统研究、人机协作）
RoomShift is a room-scale dynamic haptic environment for virtual reality, using a small swarm of robots that can move furniture. RoomShift consists of nine shape-changing robots: Roombas with mechanical scissor lifts. These robots drive beneath a piece of furniture to lift, move and place it. By augmenting virtual scenes with physical objects, users can sit on, lean against, place and otherwise interact with furniture with their whole body; just as in the real world. When the virtual scene changes or users navigate within it, the swarm of robots dynamically reconfigures the physical environmen…
The state of the art review broadly oversees the use of novel research utilized in the creation of virtual environments applied in interactive art experiences, with a specific focus on the application of procedural animation in spatially augmented reality (SAR) exhibitions. These art exhibitions frequently combine sensory displays that appeal, replace, and augment the visual, auditory and touch or haptic senses. We analyze and break down art-technology related innovations in the last three years, and thoroughly identify the most recent and vibrant applications of interactive art experiences in…
We present a mesoscale finger-mounted 4-degree-of-freedom (DoF) haptic device that is created using origami fabrication techniques. The 4-DoF device is a parallel kinematic mechanism capable of delivering normal, shear, and torsional haptic feedback to the fingertip. Traditional methods of robot fabrication are not well suited for designing small robotic devices because it is challenging and expensive to manufacture small, low-friction joints. Our device uses origami manufacturing principles to reduce complexity and the device footprint. We characterize the bandwidth, workspace, and force outp…
