arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.09211cs.AI
收藏

动态神经通信:计算机视觉与脑机接口的融合

Dynamic Neural Communication: Convergence of Computer Vision and Brain-Computer Interface

Korea University(高丽大学)

Ji-Ha Park, Seo-Hyun Lee, Soowon Kim, Seong-Whan Lee

AI总结:

本研究提出一种融合计算机视觉与脑机接口的动态神经通信方法,从神经信号解码视素以快速重建自然言语中的嘴唇运动,实现动态视觉输出。

摘要:

解读人类神经信号以解码静态言语意图(如文本或图像)和动态言语意图(如音频或视频),作为一种创新通信工具展现出巨大潜力。人类通信伴随着多种特征,如发音动作、面部表情和内部言语,所有这些都反映在神经信号中。然而,大多数研究仅生成简短或零碎的输出,而利用神经信号中的各种特征来提供信息丰富的通信仍然具有挑战性。在本研究中,我们提出了一种动态神经通信方法,该方法利用当前的计算机视觉和脑机接口技术。我们的方法从神经信号中捕捉用户意图,并在短时间步内解码视素以产生动态视觉输出。结果表明,在自然言语尝试过程中,从人类神经信号中快速捕捉和重建嘴唇运动具有潜力,通过计算机视觉与脑机接口的融合实现动态神经通信。

英文摘要:

Interpreting human neural signals to decode static speech intentions such as text or images and dynamic speech intentions such as audio or video is showing great potential as an innovative communication tool. Human communication accompanies various features, such as articulatory movements, facial expressions, and internal speech, all of which are reflected in neural signals. However, most studies only generate short or fragmented outputs, while providing informative communication by leveraging various features from neural signals remains challenging. In this study, we introduce a dynamic neural communication method that leverages current computer vision and brain-computer interface technologies. Our approach captures the user's intentions from neural signals and decodes visemes in short time steps to produce dynamic visual outputs. The results demonstrate the potential to rapidly capture and reconstruct lip movements during natural speech attempts from human neural signals, enabling dynamic neural communication through the convergence of computer vision and brain--computer interface.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑