多模态大模型

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality

Xiyun Hu, Dizhi Ma, Fengming He, Zhengzhe Zhu, Shao-Kang Hsia, Chenfei Zhu, Ziyi Liu, Karthik Ramani

专题命中 音频语音多模态 ：multimodal(abstract)

Audo-Sight: Enabling Ambient Interaction For Blind And Visually Impaired Individuals

Bhanuja Ainary

专题命中 音频语音多模态 ：multimodal(abstract)

Comments This thesis was conducted under the guidance of Mohsen Amini Salehi. Special thanks to Minseo Kim and Jacob Bradshaw for their valuable contributions and support throughout the research process. 60 pages, 13 Figures, 2 Tables

Design Probes for AI-Driven AAC: Addressing Complex Communication Needs in Aphasia

Lei Mao, Jong Ho Lee, Yasmeen Faroqi Shah, Stephanie Valencia

专题命中 音频语音多模态 ：multi-modal(abstract)

User-Centered Insights into Assistive Navigation Technologies for Individuals with Visual Impairment

Iman Soltani, Johnaton Schofield, Mehran Madani, Daniel Kish, Parisa Emami-Naeini

专题命中 音频语音多模态 ：multimodal(abstract)

Who's Watching You Zoom? Investigating Privacy of Third-Party Zoom Apps

Saharsh Goenka, Adit Prabhu, Payge Sakurai, Mrinaal Ramachandran, Rakibul Hasan

专题命中 音频语音多模态 ：audio-visual(abstract)

TransforMerger: Transformer-based Voice-Gesture Fusion for Robust Human-Robot Communication

Petr Vanc, Karla Stepanova

机构 * Czech Technical University in Prague（布拉格捷克技术大学） ； Czech Institute of Informatics, Robotics, and Cybernetics（捷克信息学、机器人学与控制论研究所） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)

Comments 8 pages, 7 figures

CyanKitten: AI-Driven Markerless Motion Capture for Improved Elderly Well-Being

Mengyao Guo, Yu Nie, Jinda Han, Zongxing Li, Ze Gao

专题命中 音频语音多模态 ：multimodal(abstract)

Comments Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, April 26-May 1, 2025, Yokohama, Japan

TartanAviation: Image, Speech, and ADS-B Trajectory Datasets for Terminal Airspace Operations

Jay Patrikar, Joao Dantas, Brady Moon, Milad Hamidi, Sourish Ghosh, Nikhil Keetha, Ian Higgins, Atharva Chandak, Takashi Yoneyama, Sebastian Scherer

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Aeronautics Institute of Technology（航空技术学院） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)

Comments 8 pages, 6 figures, 2 tables

Journal ref Scientific Data volume 12, Article number: 468 (2025)

AdaptAI: A Personalized Solution to Sense Your Stress, Fix Your Mess, and Boost Productivity

Rushiraj Gadhvi, Soham Petkar, Priyansh Desai, Shreyas Ramachandran, Siddharth Siddharth

专题命中 音频语音多模态 ：multimodal(abstract)

Comments Accepted for publication at the ACM Conference on Human Factors in Computing Systems (CHI) Late Breaking Work 2025

Actual Achieved Gain and Optimal Perceived Gain: Modeling Human Take-over Decisions Towards Automated Vehicles' Suggestions

Shuning Zhang, Xin Yi, Shixuan Li, Chuye Hong, Gujun Chen, Jiarui Liu, Xueyang Wang, Yongquan Hu, Yuntao Wang, Hewu Li

专题命中 音频语音多模态 ：multi-modal(abstract)

Innovative Framework for Early Estimation of Mental Disorder Scores to Enable Timely Interventions

Himanshi Singh, Sadhana Tiwari, Sonali Agarwal, Ritesh Chandra, Sanjay Kumar Sonbhadra, Vrijendra Singh

专题命中 音频语音多模态 ：multimodal(abstract)

TeamVision: An AI-powered Learning Analytics System for Supporting Reflection in Team-based Healthcare Simulation

Vanessa Echeverria, Linxuan Zhao, Riordan Alfredo, Mikaela Milesi, Yuequiao Jin, Sophie Abel, Jie Fan, Lixiang Yan, Xinyu Li, Samantha Dix, Rosie Wotherspoon, Hollie Jaggard, Abra Osborne, Simon Buckingham Shum, Dragan Gasevic, Roberto Martinez-Maldonado

专题命中 音频语音多模态 ：multimodal(abstract)

Comments Accepted to CHI 2025

Gender Bias and Property Taxes

Gordon Burtch, Alejandro Zentner

专题命中 音频语音多模态 ：multi-modal(abstract)

ROSAnnotator: A Web Application for ROSBag Data Analysis in Human-Robot Interaction

Yan Zhang, Haoqi Li, Ramtin Tabatabaei, Wafa Johal

机构 * University of Melbourne（墨尔本大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)

Comments Accepted to HRI 2025

Implicit Steganography Beyond the Constraints of Modality

Sojeong Song, Seoyun Yang, Chang D. Yoo, Junmo Kim

机构 * Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST）） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)

Comments 25 pages, Accepted at European Conference on Computer Vision (ECCV)

Context-Aware Deep Learning for Multi Modal Depression Detection

Genevieve Lam, Huang Dongyan, Weisi Lin

专题命中 音频语音多模态 ：multi-modal(abstract)

Comments Presented as an Oral at International Conference on Acoustics, Speech and Signal Processing 2019, United Kingdom

Re-calibrating methodologies in social media research: Challenge the visual, work with Speech

Hongrui Jin

专题命中 音频语音多模态 ：multimodal(abstract)

Comments 11 pages (excluding references), 3 figures

Stochastic Approximation and Brownian Repulsion based Evolutionary Search

Rajdeep Dutta, T Venkatesh Varma, Saikat Sarkar, Mariya Mamajiwala, Noor Awad, Senthilnath Jayavelu, Debasish Roy

专题命中 音频语音多模态 ：multi-modal(abstract)

Zeitgebers-Based User Time Perception Analysis and Data-Driven Modeling via Transformer in VR

Yi Li, Zengyu Liu, Xiandi Zhu, Ning Xie

专题命中 音频语音多模态 ：multimodal(abstract)

Comments 12pages,7 figures

Alexa, I Wanna See You: Envisioning Smart Home Assistants for the Deaf and Hard-of-Hearing

Tyrone Justin Sta. Maria, Jordan Aiko Deja

专题命中 音频语音多模态 ：multimodal(abstract)

Comments 6 pages, 1 figure, 21 references

Journal ref Proceedings of CHIRP 2024: Transforming HCI Research in the Philippines Workshop

Privacy-preserving Robotic-based Multi-factor Authentication Scheme for Secure Automated Delivery System

Yang Yang, Aryan Mohammadi Pasikhani, Prosanta Gope, Biplab Sikdar

专题命中 音频语音多模态 ：audio-visual(abstract)

Brain Treebank: Large-scale intracranial recordings from naturalistic language stimuli

Christopher Wang, Adam Uri Yaari, Aaditya K Singh, Vighnesh Subramaniam, Dana Rosenfarb, Jan DeWitt, Pranav Misra, Joseph R. Madsen, Scellig Stone, Gabriel Kreiman, Boris Katz, Ignacio Cases, Andrei Barbu

专题命中 音频语音多模态 ：multimodal(abstract)

Comments 36 pages, 17 figures; Accepted at NeurIPS Dataset and Benchmarks 2024

Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications

Nazmul Islam, Seokjoo Shin

专题命中 音频语音多模态 ：multimodal(abstract)

Journal ref IEEE Open Journal of the Communications Society, vol. 5, pp. 4207-4240, 2024

TelEdge: Haptic Tele-Communication of a Smartphone by Electro-Tactile Stimulation Through the Edges

Taiki Takami, Izumi Mizoguchi, Hiroyuki Kajimoto

专题命中 音频语音多模态 ：audio-visual(abstract)

Comments Part of proceedings of 6th International Conference AsiaHaptics 2024

ChartA11y: Designing Accessible Touch Experiences of Visualizations with Blind Smartphone Users

Zhuohao Jerry Zhang, John R. Thompson, Aditi Shah, Manish Agrawal, Alper Sarikaya, Jacob O. Wobbrock, Edward Cutrell, Bongshin Lee

专题命中 音频语音多模态 ：multimodal(abstract)

Methodology for 3D sound synthesis of directional acoustic sources by higher-order ambisonics

Philippe Thorner, Eric Bavu, Jean-Baptiste Doc, Christophe Langrenne

专题命中 音频语音多模态 ：multimodal(abstract)

Comments Internoise 2024, Aug 2024, Nantes (France), France

SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XR

Hyunsung Cho, Naveen Sendhilnathan, Michael Nebeling, Tianyi Wang, Purnima Padmanabhan, Jonathan Browder, David Lindlbauer, Tanya R. Jonker, Kashyap Todi

专题命中 音频语音多模态 ：cross-modal(abstract)

Comments UIST 2024

Generative Modeling Perspective for Control and Reasoning in Robotics

Takuma Yoneda

专题命中 音频语音多模态 ：multimodal(abstract)

Comments arXiv admin note: text overlap with arXiv:2302.12244

Recording Brain Activity While Listening to Music Using Wearable EEG Devices Combined with Bidirectional Long Short-Term Memory Networks

Jingyi Wang, Zhiqun Wang, Guiran Liu

专题命中 音频语音多模态 ：multimodal(abstract)

Comments 15 pages

Anytime, Anywhere: Human Arm Pose from Smartwatch Data for Ubiquitous Robot Control and Teleoperation

Fabian C Weigend, Shubham Sonawani, Michael Drolet, Heni Ben Amor

专题命中 音频语音多模态 ：multimodal(abstract)

Comments 8 pages, 10, figures, 1 table, conference: IROS