Recently, a lot of works show promising directions for audio design in augmented reality (AR). These works are mainly focused on how to improve user experience and make AR more realistic. But even though these improvements seem promising, these new possibilities could also be used as an input for manipulative design. This survey aims to analyze all recent discoveries in audio development regarding AR and argue what kind of "manipulative" effect this could have on the user. It can be concluded that even though there are many works explaining the effects of audio design in AR, very few works point out the risk of harm or manipulation toward the user. Future works could contain more awareness of this problem or maybe even
Augmented Reality (AR) applications are becoming more mainstream, with successful examples in the mobile environment like Pokemon GO. Current malicious techniques can exploit these environments' immersive and mixed nature (physical-virtual) to trick users into providing more personal information, i.e., dark patterns. Dark patterns are deceiving techniques (e.g., interface tricks) designed to influence individuals' behavioural decisions. However, there are few studies regarding dark patterns' potential issues in AR environments. In this work, using scenario construction to build our prototypes,…
Virtual Reality (VR) users often experience postural instability, i.e., balance problems, which could be a major barrier to universal usability and accessibility for all, especially for persons with balance impairments. Prior research has confirmed the imbalance effect, but minimal research has been conducted to reduce this effect. We recruited 42 participants (with balance impairments: 21, without balance impairments: 21) to investigate the impact of several auditory techniques on balance in VR, specifically spatial audio, static rest frame audio, rhythmic audio, and audio mapped to the cente…
Audio description, a form of trans-modal media translation, allows people who are blind or visually impaired access to visually-oriented, socio-cultural, or historical public discourse alike. Although audio description has gained more prominence in media policy and research lately, it rarely has been studied empirically. Yet this paper presents quantitative and qualitative survey data on its challenges and opportunities, through the analysis of responses from 483 participants in a national sample, with 334 of these respondents being blind. Our results give insight into audio description use in…
Social Virtual Reality (VR) is growing for remote socialization and collaboration. However, current social VR applications are not accessible to people with visual impairments (PVI) due to their focus on visual experiences. We aim to facilitate social VR accessibility by enhancing PVI's peripheral awareness of surrounding avatar dynamics. We designed VRBubble, an audio-based VR technique that provides surrounding avatar information based on social distances. Based on Hall's proxemic theory, VRBubble divides the social space with three Bubbles -- Intimate, Conversation, and Social Bubble -- gen…
Mindfulness can be defined as increased awareness of and sustained attentiveness to the present moment. Recently, there has been a growing interest in the applications of mindfulness for empirical research in wellbeing and the use of virtual reality (VR) environments and 3D interfaces as a conduit for mindfulness training. Accordingly, the current experiment investigated whether a brief VR-based mindfulness intervention could induce a greater level of state mindfulness, when compared to an audio-based intervention and control group. Results indicated two mindfulness interventions, VR-based and…
Media streaming has been adopted for a variety of applications such as entertainment, visualization, and design. Unlike video/audio streaming where the content is usually consumed sequentially, 3D applications such as gaming require streaming 3D assets to facilitate client-side interactions such as object manipulation and viewpoint movement. Compared to audio and video streaming, 3D streaming often requires larger data sizes and yet lower latency to ensure sufficient rendering quality, resolution, and latency for perceptual comfort. Thus, streaming 3D assets can be even more challenging than s…
With face-to-face music collaboration being severely limited during the recent pandemic, mixed reality technologies and their potential to provide musicians a feeling of "being there" with their musical partner can offer tremendous opportunities. In order to assess this potential, we conducted a laboratory study in which musicians made music together in real-time while simultaneously seeing their jamming partner's mixed reality point cloud via a head-mounted display and compared mental effects such as flow, affect, and co-presence to an audio-only baseline. In addition, we tracked the musician…
MR-DAW: Towards Collaborative Digital Audio Workstations in Mixed Reality
SolJAMM Research（SolJAMM研究）；University of Colorado Boulder（科罗拉多大学博尔德分校）；ATLAS Insitute（ATLAS研究所）；Hsinchu County Liou-Jia Senior High School（新竹县柳家高中）；National Taiwan Normal University（台湾师范大学）；University of Trento（特伦托大学）
The objective of this study is to investigate the impact of several auditory feedback modalities on gait (i.e., walking patterns) in virtual reality (VR). Prior research has substantiated gait disturbances in VR users as one of the primary obstacles to VR usability. However, minimal research has been done to mitigate this issue. We recruited 39 participants (with mobility impairments: 18, without mobility impairments: 21) who completed timed walking tasks in a real-world environment and the same tasks in a VR environment with various types of auditory feedback. Within-subject results showed th…
When living apart, grandparents and grandchildren often use audio-visual communication approaches to stay connected. However, these approaches seldom provide sufficient companionship and intimacy due to a lack of co-presence and spatial interaction, which can be fulfilled by immersive virtual reality (VR). To understand how grandparents and grandchildren might leverage VR to facilitate their remote communication and better inform future design, we conducted a user-centered participatory design study with twelve pairs of grandparents and grandchildren. Results show that VR affords casual and eq…
From Temporal to Spatial: Designing Spatialized Interactions with Segmented-audios in Immersive Environments for Active Engagement with Performing Arts Intangible Cultural Heritage
A cluster of research in Affective Computing suggests that it is possible to infer some characteristics of users' affective states by analyzing their electrophysiological activity in real-time. However, it is not clear how to use the information extracted from electrophysiological signals to create visual representations of the affective states of Virtual Reality (VR) users. Visualization of users' affective states in VR can lead to biofeedback therapies for mental health care. Understanding how to visualize affective states in VR requires an interdisciplinary approach that integrates psycholo…
Emotions are multifaceted phenomena that affect our behaviour, perception, and cognition. Increasing evidence indicates that induction mechanisms play a crucial role in triggering emotions by simulating the sensations required for an experimental design. Over the years, many reviews have evaluated a passive elicitation mechanism where the user is an observer, ignoring the importance of self-relevance in emotional experience. So, in response to the gap in the literature, this study intends to explore the possibility of using Virtual Reality (VR) as an active mechanism for emotion induction. Fur…
Few VR applications and games implement captioning of speech and audio cues, which either inhibits or prevents access of their application by deaf or hard of hearing (DHH) users, new language learners, and other caption users. Additionally, little to no guidelines exist on how to implement live captioning on VR headsets and how it may differ from traditional television captioning. To help fill the void of information behind user preferences of different VR captioning styles, we conducted a study with eight DHH participants to test three caption movement behaviors (headlocked, lag, and appear) …
AI Assistants for Spaceflight Procedures: Combining Generative Pre-Trained Transformer and Retrieval-Augmented Generation on Knowledge Graphs With Augmented Reality Cues
German Aerospace Center（德国航空航天中心）；AI Institute in Dynamic Systems, University of Washington（华盛顿大学动态系统人工智能研究所）；European Space Agency（欧洲空间局）
K. N. Toosi University of Technology（KN图西理工大学）；Western University（韦仕敦大学）
People suffering from Alzheimer's disease (AD) and their caregivers seek different approaches to cope with memory loss. Although AD patients want to live independently, they often need help from caregivers. In this situation, caregivers may attach notes on every single object or take out the contents of a drawer to make them visible before leaving the patient alone at home. This study reports preliminary results on an Ambient Assisted Living (AAL) real-time system, achieved through the Internet of Things (IoT) and Augmented Reality (AR) concepts, aimed at helping people suffering from AD. The …
The impact of design factors of virtual and augmented reality on tertiary students user experience in Metaverse
Faculty of College of Industrial Technology, Technological University of the Philippines（菲律宾理工大学工业技术学院）；Office of the Vice Chancellor for Academic Affairs, University of the Philippines Open University（菲律宾大学开放大学学术事务副校长办公室）
The study aims to develop an application that will serve as an alternative learning tool for learning Asian Studies. The delivery of lessons into a virtual reality game depends on the pace of students. The developed application comprises several more features that enable users to get valuable information from an immersive environment. The researchers used Rapid Application Development (RAD) in developing the application. It follows phases such as requirement planning, user design, construction, and cutover. Two sets of questionnaires were developed, one for the teachers and another for the stu…
Due to the influence of the new coronavirus disease (COVID-19), teleworking has been expanding rapidly. Although existing interactive remote working systems are convenient, they do not allow users to adjust their spatial distance to team members at will, %"Arbitrarily" is probably not the best word here. It means without apparent reason. A better expression might be "at will." and they ignore the discomfort caused by different levels of intimacy. To solve this issue, we propose a telework support system using spatial augmented reality technology. This system calibrates the space in which video…
We developed a novel virtual reality [VR] platform with 3-dimensional sounds to help improve sensory integration and visuomotor processing for postural control and fall prevention in individuals with balance problems related to sensory deficits, such as vestibular dysfunction (disease of the inner ear). The system has scenes that simulate scenario-based environments. We can adjust the intensity of the visual and audio stimuli in the virtual scenes by controlling the user interface (UI) settings. A VR headset (HTC Vive or Oculus Rift) delivers stereo display while providing real-time position a…
MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR
Google Mountain View CA USA（谷歌山景城）；University of Michigan Ann Arbor MI USA（密歇根大学安 Arbor分校）；Columbia University New York NY USA（哥伦比亚大学）；Google San Francisco CA USA（谷歌旧金山）
Humans make decisions and act alongside other humans to pursue both short-term and long-term goals. As a result of ongoing progress in areas such as computing science and automation, humans now also interact with non-human agents of varying complexity as part of their day-to-day activities; substantial work is being done to integrate increasingly intelligent machine agents into human work and play. With increases in the cognitive, sensory, and motor capacity of these agents, intelligent machinery for human assistance can now reasonably be considered to engage in joint action with humans---i.e.…
