This paper addresses a data-driven framework, modeling expert driving skills for performance-based haptic assistance using neural networks (NNs). We have built a haptic driving training simulator to collect expert driving data and to provide proper haptic feedback. We establish an expert driving skill model by training NNs with the collected data. Then, the skill model is applied to performance-based haptic assistance to provide optimized references of the steering/pedaling movements. We evaluate the skill model and its application to performance-based haptic assistance in two user experiments. The results of the first experiment demonstrate that our skill model has appropriately captured experts' steering/pedaling skills. The results of the second experiment show that our performance-based haptic assistance can help novice drivers perform steering as expert drivers, but cannot assist their pedaling performance.
Lateral assistance systems are a significant part of the currently existing Advanced Driving Assistance System (ADAS). In most cases, such systems provide intermittent audio and haptic feedback rather than continuous feedback, making it difficult for drivers to form a clear mental model. In this paper, we propose continuous visual feedback for the lateral assistance system by leveraging the Head-Up Display (HUD) alongside haptic feedback through the steering wheel. The HUD provides visualization of the risk profile underlying the haptic feedback. We hypothesize that our proposed visualization …
In this paper we introduce the combined use of Brain-Computer Interfaces (BCI) and Haptic interfaces. We propose to adapt haptic guides based on the mental activity measured by a BCI system. This novel approach is illustrated within a proof-of-concept system: haptic guides are toggled during a path-following task thanks to a mental workload index provided by a BCI. The aim of this system is to provide haptic assistance only when the user's brain activity reflects a high mental workload. A user study conducted with 8 participants shows that our proof-of-concept is operational and exploitable. R…
University at Buffalo（布法罗大学）；The University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This study investigates the effect of haptic control strategies on a subject's mental engagement during a fine motor handwriting rehabilitation task. The considered control strategies include an error-reduction (ER) and an error-augmentation (EA), which are tested on both dominant and non-dominant hand. A non-invasive brain-computer interface is used to monitor the electroencephalogram (EEG) activities of the subjects and evaluate the subject's mental engagement using the power of multiple frequency bands (theta, alpha, and beta). Statistical analysis of the effect of the control strategy on m…
Mobile telepresence robots (MTRs) have become increasingly popular in the expanding world of remote work, providing new avenues for people to actively participate in activities at a distance. However, humans operating MTRs often have difficulty navigating in densely populated environments due to limited situation awareness and narrow field-of-view, which reduces user acceptance and satisfaction. Shared autonomy in navigation has been studied primarily in static environments or in situations where only one pedestrian interacts with the robot. We present a multimodal shared autonomy approach, le…
People learn motor activities best when they are conscious of their errors and make a concerted effort to correct them. While haptic interfaces can facilitate motor training, existing interfaces are often bulky and do not always ensure post-training skill retention. Here, we describe a programmable haptic sleeve composed of textile-based electroadhesive clutches for skill acquisition and retention. We show its functionality in a motor learning study where users control a drone's movement using elbow joint rotation. Haptic feedback is used to restrain elbow motion and make users aware of their …
The Use of Gaze-Derived Confidence of Inferred Operator Intent in Adjusting Safety-Conscious Haptic Assistance
Apple Inc.（苹果公司）；Perelman School of Medicine at University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）；Honda Research Institute US（美国本田研究所）；Colorado School of Mines（科罗拉多矿业学院）
Advanced driver assistance systems have successfully reduced drivers' workloads and increased safety. On the other hand, the excessive use of such systems can impede the development of driving skills. However, there exist collaborative driver assistance systems, including shared and cooperative controls, which can promote effective collaboration between an assistance system and a human operator under appropriate system settings. Given an effective collaboration setup, we address the goal of simultaneously developing or maintaining driving skills while reducing workload. As there has been a pau…
Passive haptic learning (PHL) uses vibrotactile stimulation to train piano songs using repetition, even when the recipient of stimulation is focused on other tasks. However, many of the benefits of playing piano cannot be acquired without actively playing the instrument. In this position paper, we posit that passive haptic rehearsal, where active piano practice is assisted by separate sessions of passive stimulation, is of greater everyday use than solely PHL. We propose a study to examine the effects of passive haptic rehearsal for self-paced piano learners and consider how to incorporate pas…
Mechanical properties of biological tissue for both histological and pathological considerations are often required in disease diagnostics. Such properties can be simulated and explored with haptic technology. Development of cost effective haptic-based simulators and their introduction in the minimally invasive surgery learning cycle is still in its infancy. Receiving pretraining in a core set of surgical skills can reduce skill acquisition time and risks. We present the development of a visuo-haptic simulator module designed to train internal organs disease diagnostics through palpation. The …
California Institute of Technology（加州理工学院）；Stanford University（斯坦福大学）
Assistance during eating is essential for those with severe mobility issues or eating risks. However, dependence on traditional human caregivers is linked to malnutrition, weight loss, and low self-esteem. For those who require eating assistance, a semi-autonomous robotic platform can provide independence and a healthier lifestyle. We demonstrate an essential capability of this platform: safe, comfortable, and effective transfer of a bite-sized food item from a utensil directly to the inside of a person's mouth. Our system uses a force-reactive controller to safely accommodate the user's motio…
Skill-informed Data-driven Haptic Nudges for High-dimensional Human Motor Learning
Department of Electrical and Computer Engineering, Michigan State University（电气与计算机工程系，密歇根州立大学）；Department of Kinesiology and the Department of Mechanical Engineering, Michigan State University（运动科学系和机械工程系，密歇根州立大学）
Institute of Industrial Science, the University of Tokyo（东京大学工业科学研究所）
Haptic guidance in a shared steering assistance system has drawn significant attention in intelligent vehicle fields, owing to its mutual communication ability for vehicle control. By exerting continuous torque on the steering wheel, both the driver and support system can share lateral control of the vehicle. However, current haptic guidance steering systems demonstrate some deficiencies in assisting lane changing. This study explored a new steering interaction method, including the design and evaluation of an intention-based haptic shared steering system. Such an intention-based method can su…
Preoperative gestures include tactile sampling of the mechanical properties of biological tissue for both histological and pathological considerations. Tactile properties used in conjunction with visual cues can provide useful feedback to the surgeon. Development of novel cost effective haptic-based simulators and their introduction in the minimally invasive surgery learning cycle can absorb the learning curve for your residents. Receiving pre-training in a core set of surgical skills can reduce skill acquisition time and risks. We present the integration of a real-time surface stiffness adjus…
Institute of Industrial Science, The University of Tokyo（东京大学工业科学研究所）；Technical University of Darmstadt（达姆施塔特工业大学）
Shared steering control has been developed to reduce driver workload while keeping the driver in the control loop. A driver could integrate visual sensory information from the road ahead and haptic sensory information from the steering wheel to achieve better driving performance. Previous studies suggest that, compared with adaptive automation authority, fixed automation authority is not always appropriate with respect to human factors. This paper focuses on designing an adaptive shared steering control system via sEMG (surface electromyography) measurement from the forearm of the driver, and …
CaseMaster: Designing and Evaluating a Probe for Oral Case Presentation Training with LLM Assistance
School of Information Science and Technology ShanghaiTech University（上海科技大学信息科学与技术学院）；Department of Clinical Medicine Shanghai Clinical Research and Trial Center（上海临床研究试验中心临床医学部）
Several assistive technologies for users' operations have been recently developed. A user's sense of agency (SoA) decreases with increasing system assistance, possibly resulting in a decrease in the user's sense of fulfillment. This study aims to provide a design guideline for an assistive method to maintain and improve the sense of fulfillment with SoA. We propose a mathematical model describing the mechanisms by which the assistive method affects SoA and SoA induces a sense of fulfillment. The experience in the flow state is assumed to be a sense of fulfillment. The assistance effect on the …
Inquiry-based modeling is essential to scientific practice. However, modeling is difficult for novice scientists in part due to limited domain-specific knowledge and quantitative skills. VERA is an interactive tool that helps users construct conceptual models of ecological phenomena, run them as simulations, and examine their predictions. VERA provides cognitive scaffolding for modeling by supplying access to large-scale domain knowledge. The VERA system was tested by college-level students in two different settings: a general ecology lecture course (N=91) at a large southeastern R1 university…
