Providing pedestrians and other vulnerable road users with a clear indication about a fully autonomous vehicle status and intentions is crucial to make them coexist. In the last few years, a variety of external interfaces have been proposed, leveraging different paradigms and technologies including vehicle-mounted devices (like LED panels), short-range on-road projections, and road infrastructure interfaces (e.g., special asphalts with embedded displays). These designs were experimented in different settings, using mockups, specially prepared vehicles, or virtual environments, with heterogeneous evaluation metrics. Promising interfaces based on Augmented Reality (AR) have been proposed too, but their usability and effectiveness have not been tested yet. This paper aims to complement such body of literature by presenting a comparison of state-of-the-art interfaces and new designs under common conditions. To this aim, an immersive Virtual Reality-based simulation was developed, recreating a well-known scenario represented by pedestrians crossing in urban environments under non-regulated conditions. A user study was then performed to investigate the various dimensions of vehicle-to-pedestrian interaction leveraging objective and subjective metrics. Even though no interface clearly stood out over all the considered dimensions, one of the AR designs achieved state-of-the-art results in terms of safety and trust, at the cost of higher cognitive effort and lower intuitiveness compared to LED panels showing anthropomorphic features. Together with rankings on the various dimensions, indications about advantages and drawbacks of the various alternatives that emerged from this study could provide important information for next developments in the field.
Level of emotional arousal of one's body changes in response to external stimuli in an environment. Given the risks involved while crossing streets, particularly at unsignalized mid-block crosswalks, one can expect a change in the stress level of pedestrians. In this study, we investigate the levels and changes in pedestrian stress, under different road crossing scenarios in immersive virtual reality. To measure the stress level of pedestrians, we used Galvanic Skin Response (GSR) sensors. To collect the required data for the model, Virtual Immersive Reality Environment (VIRE) tool is used, wh…
We use an immersive virtual reality environment to explore the intricate social cues that underlie non-verbal communication involved in a pedestrian's crossing decision. We "hack" non-verbal communication between pedestrian and vehicle by engineering a set of 15 vehicle trajectories, some of which follow social conventions and some that break them. By subverting social expectations of vehicle behavior we show that pedestrians may use vehicle kinematics to infer social intentions and not merely as the state of a moving object. We investigate human behavior in this virtual world by conducting a …
Enhancing Safety in Automated Ports: A Virtual Reality Study of Pedestrian-Autonomous Vehicle Interactions under Time Pressure, Visual Constraints, and Varying Vehicle Size
Mobile navigation apps are among the most used mobile applications and are often used as a baseline to evaluate new mobile navigation technologies in field studies. As field studies often introduce external factors that are hard to control for, we investigate how pedestrian navigation methods can be evaluated in virtual reality (VR). We present a study comparing navigation methods in real life (RL) and VR to evaluate if VR environments are a viable alternative to RL environments when it comes to testing these. In a series of studies, participants navigated a real and a virtual environment usin…
This paper analyzes the distracted pedestrians' waiting time before crossing the road in three conditions: 1) not distracted, 2) distracted with a smartphone and 3) distracted with a smartphone in the presence of virtual flashing LED lights on the crosswalk as a safety measure. For the means of data collection, we adapted an in-house developed virtual immersive reality environment (VIRE). A total of 42 volunteers participated in the experiment. Participants' positions and head movements were recorded and used to calculate walking speeds, acceleration and deceleration rates, surrogate safety me…
eHMI for All -- Investigating the Effect of External Communication of Automated Vehicles on Pedestrians, Manual Drivers, and Cyclists in Virtual Reality
To ensure pedestrian friendly streets in the era of automated vehicles, reassessment of current policies, practices, design, rules and regulations of urban areas is of importance. This study investigates pedestrian crossing behaviour, as an important element of urban dynamics that is expected to be affected by the presence of automated vehicles. For this purpose, an interpretable machine learning framework is proposed to explore factors affecting pedestrians' wait time before crossing mid-block crosswalks in the presence of automated vehicles. To collect rich behavioural data, we developed a d…
A novel head-mounted virtual immersive/interactive reality environment (VIRE) is utilized to evaluate the behaviour of participants in three pedestrian road crossing conditions while 1) not distracted, 2) distracted with a smartphone, and 3) distracted with a smartphone with a virtually implemented safety measure on the road. Forty-two volunteers participated in our research who completed thirty successful (complete crossing) trials in blocks of ten trials for each crossing condition. For the two distracted conditions, pedestrians are engaged in a maze-solving game on a virtual smartphone, whi…
Occlusion-induced risk and interventions in pedestrian-autonomous truck interactions on multi-lane roads: A virtual reality study
University College London（伦敦大学学院）；Ningbo University（宁波大学）；The University of Hong Kong（香港大学）；Shanghai Maritime University（上海海事大学）；Tongji University（同济大学）
When working with emergent and appealing technologies as Virtual Reality, Mixed Reality and Augmented Reality, the issue of definitions appear very often. Indeed, our experience with various publics allows us to notice that technology definitions pose ambiguity and representation problems for informed as well as novice users. In this paper we present Immercity, a content curation system designed in the context of a collaboration between the University of Montpellier and CapGemi-ni, to deliver a technology watch. It is also used as a testbed for our experiences with Virtual, Mixed and Augmented…
We propose a framework called inverse augmented reality (IAR) which describes the scenario that a virtual agent living in the virtual world can observe both virtual objects and real objects. This is different from the traditional augmented reality. The traditional virtual reality, mixed reality and augmented reality are all generated for humans, i.e., they are human-centered frameworks. On the contrary, the proposed inverse augmented reality is a virtual agent-centered framework, which represents and analyzes the reality from a virtual agent's perspective. In this paper, we elaborate the frame…
When establishing a visual connection between a virtual reality user and an augmented reality user, it is important to consider whether the augmented reality user faces a surplus of information. Augmented reality, compared to virtual reality, involves two, not one, planes of information: the physical and the virtual. We propose SelectVisAR, a selective visualisation system of virtual environments in augmented reality. Our system enables an augmented reality spectator to perceive a co-located virtual reality user in the context of four distinct visualisation conditions: Interactive, Proximity, …
In this paper, we propose CaminAR, an augmented reality (AR) system for remote social walking among dyads. CaminAR enables two people who are physically away from one another to synchronously see and hear each other while going on a walk. The system shows a partner's avatar superimposed onto the physical world using smart glasses while walking and talking. Using a combination of visual and auditory augments, CaminAR simulates the experience of co-located walking when dyads are apart.
Malmö University（马尔默大学）；Lund University（隆德大学）；University West（西部大学）
Augmented reality (AR) games are a rich environment for researching and testing computational systems that provide subtle user guidance and training. In particular computer systems that aim to augment a user's situation awareness benefit from the range of sensors and computing power available in AR headsets. In this work-in-progress paper, we present a new environment for research into situation awareness and attention guidance (SAAG): an augmented reality version of the board game Carcassonne. We also present our initial work in producing a SAAG pipeline, including the creation of game state …
Augmented Reality (AR) is becoming mobile. Mobile devices have many constraints but also rich new features that traditional desktop computers do not have. There are several survey papers on AR, but none is dedicated to Mobile Augmented Reality (MAR). Our work serves the purpose of closing this gap. The contents are organized with a bottom-up approach. We first present the state-of-the-art in system components including hardware platforms, software frameworks and display devices, follows with enabling technologies such as tracking and data management. We then survey the latest technologies and …
Using augmented reality in education is already a common concept, as it has the potential to turn learning into a motivational learning experience. However, current research only covers the students site of learning. Almost no research focuses on the teachers' site and whether augmented reality could potentially improve his/her workflow of teaching the students or not. Many researchers do not differentiate between multiple user roles, like a student and a teacher. To allow investigation into these lacks of research, a teaching system "Augmentix" is presented, which includes a differentiation b…
Navigation has been a popular area of research in both academia and industry. Combined with maps, and different localization technologies, navigation systems have become robust and more usable. By combining navigation with augmented reality, it can be improved further to become realistic and user friendly. This paper surveys existing researches carried out in this area, describes existing techniques for building augmented reality navigation systems, and the problems faced.
This document summarizes the major milestones in mobile Augmented Reality between 1968 and 2014. Major parts of the list were compiled by the member of the Christian Doppler Laboratory for Handheld Augmented Reality in 2010 (author list in alphabetical order) for the ISMAR society. Later in 2013 it was updated, and more recent work was added during preparation of this report. Permission is granted to copy and modify.
AI Assistants for Spaceflight Procedures: Combining Generative Pre-Trained Transformer and Retrieval-Augmented Generation on Knowledge Graphs With Augmented Reality Cues
German Aerospace Center（德国航空航天中心）；AI Institute in Dynamic Systems, University of Washington（华盛顿大学动态系统人工智能研究所）；European Space Agency（欧洲空间局）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Colorado School of Mines（科罗拉多矿业学院）；IBM Research AI（IBM人工智能研究院）
Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) has been gaining considerable attention in research in recent years. However, the HRI community lacks a set of shared terminology and framework for characterizing aspects of mixed reality interfaces, presenting serious problems for future research. Therefore, it is important to have a common set of terms and concepts that can be used to precisely describe and organize the diverse array of work being done within the field. In this paper, we present a novel taxonomic framework for different types of VAM-HRI interfaces, c…
Augmented Reality consists of merging live images with virtual layers of information. The rapid growth in the popularity of smartphones and tablets over recent years has provided a large base of potential users of Augmented Reality technology, and virtual layers of information can now be attached to a wide variety of physical objects. In this article, we explore the potential of Augmented Reality for astrophysical research with two distinct experiments: (1) Augmented Posters and (2) Augmented Articles. We demonstrate that the emerging technology of Augmented Reality can already be used and imp…
Augmented Reality, or simply AR, is the incorporation of information in digital format that includes live footage of a certain user's real-time environment. Also now, various universities are using Augmented Reality. Applying the technology in the education sector can result in having a smart campus. In line with that, this paper will discuss how Augmented Reality is being used now in different learning areas.
The definition of the augmented reality concept is based on the analysis of scientific publications. It is noted that online experiments with augmented reality provide students with the opportunity to observe and describe the operation with real systems by changing their parameters, and also partially replace experimental installations with objects of augmented reality. The scheme for realizing the augmented reality is considered. The possibilities of working with augmented reality objects in teaching physics is highlighted. It is indicated that the use of the augmented reality tools allows to…
Augmented Reality (AR) is an emerging field ripe for experimentation, especially when it comes to developing the kinds of applications and experiences that will drive mass adoption of the technology. While we aren't aware of any current consumer product that realize a wearable, wide Field of View (FoV), AR Head Mounted Display (HMD), such devices will certainly come. In order for these sophisticated, likely high-cost hardware products to succeed, it is important they provide a high quality user experience. To that end, we prototyped 4 experimental applications for wide FoV displays that will l…
In recent years, Augmented Reality (AR) and Virtual Reality (VR) have gained considerable commercial traction, with Facebook acquiring Oculus VR for \$2 billion, Magic Leap attracting more than \$500 million of funding, and Microsoft announcing their HoloLens head-worn computer. Where is humanity headed: a brave new dystopia-or a paradise come true? In this article, we present discussions, which started at the symposium "Making Augmented Reality Real", held at Nara Institute of Science and Technology in August 2014. Ten scientists were invited to this three-day event, which started with a full…
Virtual and augmented reality communication platforms are seen as promising modalities for next-generation remote face-to-face interactions. Our study attempts to explore non-verbal communication features in relation to their conversation context for virtual and augmented reality mediated communication settings. We perform a series of user experiments, triggering nine conversation tasks in 4 settings, each containing corresponding non-verbal communication features. Our results indicate that conversation types which involve less emotional engagement are more likely to be acceptable in virtual r…
Understanding electronics is a critical area in the maker scene. Many of the makers' projects require electronics knowledge to connect microcontrollers with sensors and actuators. Yet, learning electronics is challenging, as internal component processes remain invisible, and students often fear personal harm or component damage. Augmented Reality (AR) applications are developed to support electronics learning and visualize complex processes. This paper reflects on related work around AR and electronics that characterize open research challenges around the four characteristics functionality, fi…
Recently, a lot of works show promising directions for audio design in augmented reality (AR). These works are mainly focused on how to improve user experience and make AR more realistic. But even though these improvements seem promising, these new possibilities could also be used as an input for manipulative design. This survey aims to analyze all recent discoveries in audio development regarding AR and argue what kind of "manipulative" effect this could have on the user. It can be concluded that even though there are many works explaining the effects of audio design in AR, very few works poi…
This bachelor's thesis describes the conception and implementation of an augmented reality application for the Android platform. The intention is to demonstrate some possibilities of interaction within an augmented reality environment on mobile devices. For that purpose, a 3D-model is displayed on the devices' touchscreen using marker-based tracking. This enables the user to translate, rotate or scale the model as he wishes. He can additionally select and highlight preassigned parts of the model to display specific information for that element. To assist developers in modifying the application…
The purpose of this study is to present a comprehensive review of the use of structural equation modeling (SEM) in augmented reality (AR) studies in the context of the COVID-19 pandemic. IEEE Xplore Scopus, Wiley Online Library, Emerald Insight, and ScienceDirect are the main five data sources for data collection from Jan 2020 to May 2021. The results showed that a variety of external factors were used to construct the SEM models rather than using the parsimonious ones. The reports showed a fair balance between the direct and indirect methods to contact participants. Despite the COVID-19 pande…
We present a user interface (UI) based on augmented reality (AR) with head-mounted display (HMD) for improving situational awareness during critical operation and improve human efficiency on operations. The UI displays contextual information as well as accepts orders given from the headset to control unmanned aerial vehicles (UAVs) for assisting the rescue team. We established experiments where people had been put in a stressful situation and are asked to resolve a complex mission using a headset and a computer. Comparing both technologies, our results show that augmented reality has the poten…
As augmented reality technology and hardware become more mature and affordable, researchers have been exploring more intuitive and discoverable interaction techniques for immersive environments. In this paper, we investigate multimodal interaction for 3D object manipulation in a multi-object virtual environment. To identify the user-defined gestures, we conducted an elicitation study involving 24 participants for 22 referents with an augmented reality headset. It yielded 528 proposals and generated a winning gesture set with 25 gestures after binning and ranking all gesture proposals. We found…
This paper presents and analyses existing taxonomies of virtual and augmented reality and demonstrates knowledge gaps and mixed terminology which may cause confusion among educators, researchers, and developers. Several such occasions of confusion are presented. A methodology is then presented to construct a taxonomy of virtual reality and augmented reality applications based on a combination of: a faceted analysis approach for the overall design of the taxonomy; an existing taxonomy of educational objectives to derive the educational purpose; an information systems analysis to establish impor…
We present an early study designed to analyze how city planning and the health of senior citizens can benefit from the use of augmented reality (AR) using Microsoft's HoloLens. We also explore whether AR and VR can be used to help city planners receive real-time feedback from citizens, such as the elderly, on virtual plans, allowing for informed decisions to be made before any construction begins.
Augmented Reality (AR) and Mobile Augmented Reality (MAR) applications have gained much research and industry attention these days. The mobile nature of MAR applications limits users' interaction capabilities such as inputs, and haptic feedbacks. This survey reviews current research issues in the area of human computer interaction for MAR and haptic devices. The survey first presents human sensing capabilities and their applicability in AR applications. We classify haptic devices into two groups according to the triggered sense: cutaneous/tactile: touch, active surfaces, and mid-air, kinesthet…
This is the preprint version of our paper on Personal and Ubiquitous Computing. There is an increasing interest in creating pervasive games based on emerging interaction technologies. In order to develop touch-less, interactive and augmented reality games on vision-based wearable device, a touch-less motion interaction technology is designed and evaluated in this work. Users interact with the augmented reality games with dynamic hands/feet gestures in front of the camera, which triggers the interaction event to interact with the virtual object in the scene. Three primitive augmented reality ga…
Mindscape: Research of high-information density street environments based on electroencephalogram recording and virtual reality head-mounted simulation
Augmented Reality is the fusion of virtual components and our real surroundings. The simultaneous visibility of generated and natural objects often requires users to direct their selective attention to a specific target that is either real or virtual. In this study, we investigated whether this target is real or virtual by using machine learning techniques to classify electroencephalographic (EEG) data collected in Augmented Reality scenarios. A shallow convolutional neural net classified 3 second data windows from 20 participants in a person-dependent manner with an average accuracy above 70\…
The contributions of this paper are: (1) a taxonomy of the "Realities" (Virtual, Augmented, Mixed, Mediated, etc.), and (2) some new kinds of "reality" that come from nature itself, i.e. that expand our notion beyond synthetic realities to include also phenomenological realities. VR (Virtual Reality) replaces the real world with a simulated experience (virtual world). AR (Augmented Reality) allows a virtual world to be experienced while also experiencing the real world at the same time. Mixed Reality provides blends that interpolate between real and virtual worlds in various proportions, along…
Conventional HVAC control systems are usually incognizant of the physical structures and materials of buildings. These systems merely follow pre-set HVAC control logic based on abstract building thermal response models, which are rough approximations to true physical models, ignoring dynamic spatial variations in built environments. To enable more accurate and responsive HVAC control, this paper introduces the notion of "self-aware" smart buildings, such that buildings are able to explicitly construct physical models of themselves (e.g., incorporating building structures and materials, and the…
