For the optimum design of a driver-automation shared control system, an understanding of driver behavior based on measurements and modeling is crucial early in the development process. This paper presents a driver model through a weighting process of visual guidance from the road ahead and haptic guidance from a steering system for a lane-following task. The proposed weighting process describes the interaction of a driver with the haptic guidance steering and the driver reliance on it. A driving simulator experiment is conducted to identify the model parameters for driving manually and with haptic guidance. The proposed driver model matched the driver input torque with a satisfactory goodness of fit among fourteen participants after considering the individual differences. The validation results reveal that the simulated trajectory effectively followed the driving course by matching the measured trajectory, thereby indicating that the proposed driver model is capable of predicting driver behavior during haptic guidance. Furthermore, the effect of different degrees of driver reliance on driving performance is evaluated considering various driver states and with system failure via numerical analysis. The model evaluation results reveal the potential of the proposed driver model to be applied in the design and evaluation of a haptic guidance system.
The thesis presents contributions made to the evaluation and design of a haptic guidance system on improving driving performance in cases of normal and degraded visual information, which are based on behavior experiments, modeling and numerical simulations. The effect of shared control on driver behavior in cases of normal and degraded visual information has been successfully evaluated experimentally and numerically. The evaluation results indicate that the proposed haptic guidance system is capable of providing reliable haptic information, and is effective on improving lane following performa…
Institute of Industrial Science, The University of Tokyo（东京大学工业科学研究所）；Technical University of Darmstadt（达姆施塔特工业大学）
Shared steering control has been developed to reduce driver workload while keeping the driver in the control loop. A driver could integrate visual sensory information from the road ahead and haptic sensory information from the steering wheel to achieve better driving performance. Previous studies suggest that, compared with adaptive automation authority, fixed automation authority is not always appropriate with respect to human factors. This paper focuses on designing an adaptive shared steering control system via sEMG (surface electromyography) measurement from the forearm of the driver, and …
Corpy & Co., Inc.（Corpy公司）；The University of Tokyo（东京大学）
Driver distraction is a well-known cause for traffic collisions worldwide. Studies have indicated that shared steering control, which actively provides haptic guidance torque on the steering wheel, effectively improves the performance of distracted drivers. Recently, adaptive shared steering control based on the physiological status of the driver has been developed, although its effect on distracted driver behavior remains unclear. To this end, a high-fidelity driving simulator experiment was conducted involving 18 participants performing double lane changes. The experimental conditions compri…
Institute of Industrial Science, the University of Tokyo（东京大学工业科学研究所）
Haptic guidance in a shared steering assistance system has drawn significant attention in intelligent vehicle fields, owing to its mutual communication ability for vehicle control. By exerting continuous torque on the steering wheel, both the driver and support system can share lateral control of the vehicle. However, current haptic guidance steering systems demonstrate some deficiencies in assisting lane changing. This study explored a new steering interaction method, including the design and evaluation of an intention-based haptic shared steering system. Such an intention-based method can su…
In conditional automation, a response from the driver is expected when a take over request is issued due to unexpected events, emergencies, or reaching the operational design domain boundaries. Cooperation between the automated driving system and the driver can help to guarantee a safe and pleasant transfer if the driver is guided through a haptic guidance system that applies a slight counter-steering force to the steering wheel. We examine in this work the impact of haptic guidance systems on driving performance after a take over request was triggered to avoid sudden obstacles on the road. We…
Haptic Guidance and Haptic Error Amplification in a Virtual Surgical Robotic Training Environment
Stanford University（斯坦福大学）；Hospital for Special Surgery（特种外科医院）；University of Notre Dame（圣母大学）；University of Washington（华盛顿大学）；Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Intuitive Surgical Inc.（直觉外科公司）；Case Western Reserve University（凯斯西储大学）
It remains uncertain regarding the safety of driving in autonomous vehicles that, after a long, passive control and inattention to the driving situation, how the drivers will be effectively informed to take-over the control in emergency. In particular, the active role of vehicle force feedback on the driver's risk perception on curves has not been fully explored. To investigate it, the current paper examined the driver's cognitive and visual responses to the whole-body haptic feedback during curve negotiations. The effects of force feedback on drivers' responses on curves were investigated in …
Pohang University of Science and Technology（浦项科技大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Samsung research（三星研究院）
This paper addresses a data-driven framework, modeling expert driving skills for performance-based haptic assistance using neural networks (NNs). We have built a haptic driving training simulator to collect expert driving data and to provide proper haptic feedback. We establish an expert driving skill model by training NNs with the collected data. Then, the skill model is applied to performance-based haptic assistance to provide optimized references of the steering/pedaling movements. We evaluate the skill model and its application to performance-based haptic assistance in two user experiments…
The recently advanced robotics technology enables robots to assist users in their daily lives. Haptic guidance (HG) improves users' task performance through physical interaction between robots and users. It can be classified into optimal action-based HG (OAHG), which assists users with an optimal action, and user prediction-based HG (UPHG), which assists users with their next predicted action. This study aims to understand the difference between OAHG and UPHG and propose a combined HG (CombHG) that achieves optimal performance by complementing each HG type, which has important implications for…
In this research, we have developed a haptic situational awareness device that presents users with directional cues through continuous vibrotactile sensations. Using the device, we present user studies on the effectiveness of a torso-mounted haptic display in enhancing human situational awareness, when visual/auditory senses are degraded. A haptic display has been developed which generates continuous cues using tactile illusions and programmed to generate navigation commands. Participants are given navigation tasks where they may receive commands through any of the seven sensory modality combi…
Mobile telepresence robots (MTRs) have become increasingly popular in the expanding world of remote work, providing new avenues for people to actively participate in activities at a distance. However, humans operating MTRs often have difficulty navigating in densely populated environments due to limited situation awareness and narrow field-of-view, which reduces user acceptance and satisfaction. Shared autonomy in navigation has been studied primarily in static environments or in situations where only one pedestrian interacts with the robot. We present a multimodal shared autonomy approach, le…
We propose a system to deliver dynamic guidance in drawing, sketching and handwriting tasks via an electromagnet moving underneath a high refresh rate pressure sensitive tablet. The system allows the user to move the pen at their own pace and style and does not take away control. The system continously and iteratively measures the pen motion and adjusts magnet position and power according to the user input in real-time via a receding horizon optimal control formulation. The optimization is based on a novel approximate electromagnet model that is fast enough for use in real-time methods, yet pr…
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Centre for Meaningful Human Control（有意义的人控中心）；Department of Cognitive Robotics（认知机器人学系）；Department of Transport & Planning（交通与规划系）；Department of Intelligent Systems（智能系统系）；Delft University of Technology（代尔夫特理工大学）
Lateral assistance systems are a significant part of the currently existing Advanced Driving Assistance System (ADAS). In most cases, such systems provide intermittent audio and haptic feedback rather than continuous feedback, making it difficult for drivers to form a clear mental model. In this paper, we propose continuous visual feedback for the lateral assistance system by leveraging the Head-Up Display (HUD) alongside haptic feedback through the steering wheel. The HUD provides visualization of the risk profile underlying the haptic feedback. We hypothesize that our proposed visualization …
We present advancements in the design and development of in-vehicle infotainment systems that utilize gesture input and ultrasonic mid-air haptic feedback. Such systems employ state-of-the-art hand tracking technology and novel haptic feedback technology and promise to reduce driver distraction while performing a secondary task therefore cutting the risk of road accidents. In this paper, we document design process considerations during the development of a mid-air haptic gesture-enabled user interface for human-vehicle-interactions. This includes an online survey, business development insights…
Traffic jams occurring on highways cause increased travel time as well as increased fuel consumption and collisions. Traffic jams without a clear cause, such as an on-ramp or an accident, are called phantom traffic jams and are said to make up 50% of all traffic jams. They are the result of an unstable traffic flow caused by human driving behavior. Automating the longitudinal vehicle motion of only 5% of all cars in the flow can dissipate phantom traffic jams. However, driving automation introduces safety issues when human drivers need to take over the control from the automation. We investiga…
Haptic interfaces have untapped the sense of touch to assist multimodal music learning. We have recently seen various improvements of interface design on tactile feedback and force guidance aiming to make instrument learning more effective. However, most interfaces are still quite static; they cannot yet sense the learning progress and adjust the tutoring strategy accordingly. To solve this problem, we contribute an adaptive haptic interface based on the latest design of haptic flute. We first adopted a clutch mechanism to enable the interface to turn on and off the haptic control flexibly in …
Advanced driver assistance systems have successfully reduced drivers' workloads and increased safety. On the other hand, the excessive use of such systems can impede the development of driving skills. However, there exist collaborative driver assistance systems, including shared and cooperative controls, which can promote effective collaboration between an assistance system and a human operator under appropriate system settings. Given an effective collaboration setup, we address the goal of simultaneously developing or maintaining driving skills while reducing workload. As there has been a pau…
With the development of advanced communication technology, connected vehicles become increasingly popular in our transportation systems, which can conduct cooperative maneuvers with each other as well as road entities through vehicle-to-everything communication. A lot of research interests have been drawn to other building blocks of a connected vehicle system, such as communication, planning, and control. However, less research studies were focused on the human-machine cooperation and interface, namely how to visualize the guidance information to the driver as an advanced driver-assistance sys…
Center touchscreens are the main HMI (Human-Machine Interface) between the driver and the vehicle. They are becoming, larger, increasingly complex and replace functions that could previously be controlled using haptic interfaces. To ensure that touchscreen HMI can be operated safely, they are subject to strict regulations and elaborate test protocols. Those methods and user trials require fully functional prototypes and are expensive and time-consuming. Therefore it is desirable to estimate the workload of specific interfaces or interaction sequences as early as possible in the development pro…
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Toyota Motor North America, InfoTech Labs（丰田北美汽车公司信息技术实验室）；Columbia University（哥伦比亚大学）
With the rapid development of intelligent vehicles and Advanced Driving Assistance Systems (ADAS), a mixed level of human driver engagements is involved in the transportation system. Visual guidance for drivers is essential under this situation to prevent potential risks. To advance the development of visual guidance systems, we introduce a novel sensor fusion methodology, integrating camera image and Digital Twin knowledge from the cloud. Target vehicle bounding box is drawn and matched by combining results of object detector running on ego vehicle and position information from the cloud. The…
Modern Advanced Driver Assistance Systems (ADAS) are limited in their ability to consider the drivers intention, resulting in unnatural guidance and low customer acceptance. In this research, we focus on a novel data-driven approach to predict driver steering torque. In particular, driver behavior is modeled by learning the parameters of a Hidden Markov Model (HMM) and estimation is performed with Gaussian Mixture Regression (GMR). An extensive parameter selection framework enables us to objectively select the model hyper-parameters and prevents overfitting. The final model behavior is optimiz…
Introducing ROADS: A Systematic Comparison of Remote Control Interaction Concepts for Automated Vehicles at Road Works
Ulm University（乌尔姆大学）；UCL Interaction Centre（伦敦大学学院交互中心）；RISE Research Institutes of Sweden（瑞典RISE研究院）；University of Salzburg（萨尔茨堡大学）；AIT Austrian Institute of Technology（奥地利技术研究院）
