Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance
Computer Vision for Human-Computer Interaction Lab（人机交互计算机视觉实验室）；Center for Digital Accessibility and Assistive Technology, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院数字无障碍与辅助技术中心）
Transparent objects, such as glass walls and doors, constitute architectural obstacles hindering the mobility of people with low vision or blindness. For instance, the open space behind glass doors is inaccessible, unless it is correctly perceived and interacted with. However, traditional assistive technologies rarely cover the segmentation of these safety-critical transparent objects. In this paper, we build a wearable system with a novel dual-head Transformer for Transparency (Trans4Trans) perception model, which can segment general- and transparent objects. The two dense segmentation results are further combined with depth information in the system to help users navigate safely and assist them to negotiate transparent obstacles. We propose a lightweight Transformer Parsing Module (TPM) to perform multi-scale feature interpretation in the transformer-based decoder. Benefiting from TPM, the double decoders can perform joint learning from corresponding datasets to pursue robustness, meanwhile maintain efficiency on a portable GPU, with negligible calculation increase. The entire Trans4Trans model is constructed in a symmetrical encoder-decoder architecture, which outperforms state-of-the-art methods on the test sets of Stanford2D3D and Trans10K-v2 datasets, obtaining mIoU of 45.13% and 75.14%, respectively. Through a user study and various pre-tests conducted in indoor and outdoor scenes, the usability and reliability of our assistive system have been extensively verified. Meanwhile, the Tran4Trans model has outstanding performances on driving scene datasets. On Cityscapes, ACDC, and DADA-seg datasets corresponding to common environments, adverse weather, and traffic accident scenarios, mIoU scores of 81.5%, 76.3%, and 39.2% are obtained, demonstrating its high efficiency and robustness for real-world transportation applications.
Learning Navigation by Visual Localization and Trajectory Prediction
Institute of Mathematics of the Romanian Academy（罗马尼亚科学院数学研究所）；Faculty of Automatic Control and Computers, University Politehnica of Bucharest（布加勒斯特理工大学自动控制与计算机学院）
Expert Consensus on Criteria for the Automated Assessment of Laparoscopic Camera Navigation
University Medical Center Göttingen（哥廷根大学医学中心）；Fraunhofer Research Institution for Individualized and Cell-based Medical Engineering (IMTE)（弗劳恩霍夫个性化与细胞医学工程研究所）
Goal-conditioned reinforcement learning for ultrasound navigation guidance
King’s College London（伦敦国王学院）；Siemens Healthcare Limited（西门子医疗有限公司）；Siemens Healthineers（西门子医疗）；Siemens Healthineers AG（西门子医疗股份公司）；Carol Davila University of Medicine and Pharmacy Bucharest（布加勒斯特卡罗尔·达维拉医药大学）；Fortis Institute of Medical Sciences（富蒂斯医学院）；Guy’s and St Thomas’ NHS Foundation Trust（盖伊和圣托马斯国民保健信托基金）
VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation
School of Computing（计算机学院）；Clemson University（克莱姆森大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Stevens Institute of Technology（史蒂文斯理工学院）；School of Cultural and Social Studies（文化与社会研究学院）；Tokai University（东海大学）
AMAVA: Adaptive Motion-Aware Video-to-Audio Framework for Visually-Impaired Assistance
Department of Computer Science, San Francisco State University（计算机科学系，旧金山州立大学）；Department of Mathematics, San Francisco State University（数学系，旧金山州立大学）；Department of Computer Engineering, San Francisco State University（计算机工程系，旧金山州立大学）
Real-Time Glass Detection and Reprojection using Sensor Fusion Onboard Aerial Robots
ARL DCIST CRA；NSF Grant CCR-2112665（NSF CCR-2112665）；NSF Graduate Research Fellowship Grant No. DGE-1845298（NSF DGE-1845298）；IoT4Ag Engineering Research Center
Scene-Aware Vectorized Memory Multi-Agent Framework with Cross-Modal Differentiated Quantization VLMs for Visually Impaired Assistance
School of Information；Software Engineering, University of Electronic Science；Sichuan Provincial Key Laboratory for Human Disease Gene Study, Sichuan Academy of Medical Sciences \& Sichuan Provincial People's Hospital, University of Electronic Science；Faculty of Computing, Harbin Institute of Technology, Harbin, China
Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions
School of Navigation, Wuhan University of Technology（武汉理工大学航海学院）；Hong Kong Polytechnic University（香港理工大学）；School of Cyber Science and Technology, Sun Yat-sen University at Shenzhen（中山大学深圳校区网络空间科学与技术学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation
Beijing Institute of Technology（北京理工大学）；ETH Zurich（苏黎世联邦理工学院）；Zhejiang University（浙江大学）；Yangtze Delta Region Academy of Beijing Institute of Technology（北京理工大学长三角研究院）
Can Foundation Models Reliably Identify Spatial Hazards? A Case Study on Curb Segmentation
NYU Tandon School of Engineering（纽约大学坦登工程学院）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；NYU Langone Health（纽约大学朗格尼健康中心）；New York University Grossman School of Medicine（纽约大学格罗斯曼医学院）
