arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.05545eess.AS
收藏

多模态深度学习方法用于实时空间房间脉冲响应计算

Multimodal Deep Learning Method for Real-Time Spatial Room Impulse Response Computing

Beijing Institute of Technology(北京理工大学)

Zhiyu Li, Xinwen Yue, Shenghui Zhao, Jing Wang

AI总结:

本文提出一种多模态深度学习模型,用于VR混响生成实时空间房间脉冲响应,通过几何声学计算低阶反射,提升计算效率与个性化声学感知重建能力。

摘要:

我们提出了一种用于VR混响的多模态深度学习模型,该模型能实时生成空间房间脉冲响应(SRIR),以重建特定场景的听觉感知。将SRIR作为输出可降低计算复杂度并便于与个性化头部相关转移函数集成。该模型输入两种模态:场景信息和波形,其中波形对应低阶反射(LoR)。LoR可通过几何声学(GA)高效计算,但对深度学习模型而言仍难以准确预测。首先利用场景几何、声学特性、源坐标和听众坐标通过GA实时计算LoR,并将LoR和这些特征作为模型输入。构建了一个包含多个场景及其相应SRIR的新数据集。该数据集具有更大的多样性。实验结果表明所提模型具有优越的性能。

英文摘要:

We propose a multimodal deep learning model for VR auralization that generates spatial room impulse responses (SRIRs) in real time to reconstruct scene-specific auditory perception. Employing SRIRs as the output reduces computational complexity and facilitates integration with personalized head-related transfer functions. The model takes two modalities as input: scene information and waveforms, where the waveform corresponds to the low-order reflections (LoR). LoR can be efficiently computed using geometrical acoustics (GA) but remains difficult for deep learning models to predict accurately. Scene geometry, acoustic properties, source coordinates, and listener coordinates are first used to compute LoR in real time via GA, and both LoR and these features are subsequently provided as inputs to the model. A new dataset was constructed, consisting of multiple scenes and their corresponding SRIRs. The dataset exhibits greater diversity. Experimental results demonstrate the superior performance of the proposed model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑