AMAVA: Adaptive Motion-Aware Video-to-Audio Framework for Visually-Impaired Assistance
Department of Computer Science, San Francisco State University（计算机科学系，旧金山州立大学）；Department of Mathematics, San Francisco State University（数学系，旧金山州立大学）；Department of Computer Engineering, San Francisco State University（计算机工程系，旧金山州立大学）
Benjamin Klein, Kazi Ruslan Rahman, Sanchita Ghose
备注：
8 pages, 7 figures. Published in the Proceedings of the 15th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2026), pages 282--289
发表信息：
In Proceedings of the 15th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2026), pages 282--289, 2026
Navigational aids for blind and low vision individuals struggle conveying dynamic real-world environments, leading to cognitive overload from continuous, undifferentiated feedback. We present AMAVA, a novel real-time video-to-audio framework that converts mobile device video into contextually relevant sound effects or text-to-speech descriptions. We propose a motion-aware pipeline using a lightweight AI classification model to distinguish between low and high-movement scenes followed by a real-time text-to-audio synthesis pipeline to enhance environmental perception more efficiently. In static environments, AMAVA generates spoken audio scene descriptions for situational awareness. In high-movement situations, it prioritizes safety by delivering sound cues, such as spoken hazard alerts and environmental sound effects. These audio outputs are produced by a decoder-only transformer-based vision-language model with mixture-of-experts and cross-modal attention for visual understanding, in conjunction with neural text-to-speech and natural sound synthesis networks. The proposed framework uses prompt-based caching and category-specific throttling to avoid auditory clutter and minimize latency. We present a comprehensive evaluation of the system, including a real-time navigation study comparing a white cane alone versus with AMAVA, that shows a significant increase in user confidence and perceived safety.
AI-based Wearable Vision Assistance System for the Visually Impaired: Integrating Real-Time Object Recognition and Contextual Understanding Using Large Vision-Language Models
Scene-Aware Vectorized Memory Multi-Agent Framework with Cross-Modal Differentiated Quantization VLMs for Visually Impaired Assistance
School of Information；Software Engineering, University of Electronic Science；Sichuan Provincial Key Laboratory for Human Disease Gene Study, Sichuan Academy of Medical Sciences \& Sichuan Provincial People's Hospital, University of Electronic Science；Faculty of Computing, Harbin Institute of Technology, Harbin, China
EgoBlind: Towards Egocentric Visual Assistance for the Blind
National University of Singapore（新加坡国立大学）；Communication University of China（中国传媒大学）；University of Science and Technology of China（中国科学技术大学）；Hefei University of Technoloy（合肥工业大学）
Vision-based Wearable Steering Assistance for People with Impaired Vision in Jogging
Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）；School of Information and Electronic, Beijing Institute of Technology（北京理工大学信息与电子学院）；School of Mechanical Engineering, Tongji University（同济大学机械工程学院）
本文提出部署于Jetson Orin NX的视觉可穿戴转向辅助系统，通过轻量多任务网络检测跑道线与障碍物，并结合采样和样条曲线规划，帮助视障者安全慢跑。
KinTwin: Imitation Learning with Torque and Muscle Driven Biomechanical Models Enables Precise Replication of Able-Bodied and Impaired Movement from Markerless Motion Capture
Department of PM&R Northwestern University Shirley Ryan AbilityLab（运动医学系西北大学施里尔 Ryan 能力实验室）
