基于神经算子的头部相关传递函数(HRTF)上采样与个性化
HRTF Upsampling and Personalization via Neural Operators
AI总结:
本研究针对个性化双耳音频中HRTF稀疏测量上采样的挑战,提出神经算子SPARO,在SONICOM、HUTUBS数据集上实现最优LSD、ILD误差,重建速度远超RANF。
摘要:
从稀疏测量值进行头部相关传递函数(HRTF)空间上采样是个性化双耳音频领域的核心挑战。我们将HRTF上采样问题建模为算子学习问题,并提出SPARO——一种神经算子,可将一组空间方向上的稀疏测量值映射为HRTF场,这是一种可在任意位置查询的连续表示。SPARO将测量值投影到一组学习得到的球谐潜模态上,通过Transformer块处理这些潜模态,再在查询方向上解码得到的表示。在SONICOM数据集上,采用LAP24测量配置时,SPARO在3到100个测量值的所有稀疏度水平下均取得最优的对数谱失真(LSD)和耳间级差(ILD)误差:在100个测量值时,其LSD比最强基线低18%,ILD误差低43%,同时耳间时间差(ITD)误差也具有竞争力。它重建密集HRTF的速度比领先的神经场方法RANF快15至373倍。我们进一步在HUTUBS数据集上验证了该方法,证明其在不同HRTF数据集上均有效。
英文摘要:
Head-related transfer function (HRTF) spatial upsampling from sparse measurements is a fundamental challenge in personalized binaural audio. We formulate HRTF upsampling as an operator learning problem and introduce SPARO, a neural operator that maps sparse measurements at a set of spatial directions to the HRTF field, a continuous representation that can be queried at arbitrary positions. SPARO projects the measurements onto a learned set of spherical latent modes, processes the latent modes with Transformer blocks, and decodes the resulting representations at query directions. On SONICOM, using the LAP24 measurement configuration, SPARO attains the best log-spectral distortion (LSD) and interaural level difference (ILD) error at every sparsity level from 3 to 100 measurements (18% lower LSD and 43% lower ILD error than the strongest baseline at 100 measurements) while remaining competitive in interaural time difference (ITD) error. It reconstructs a dense HRTF 15 to 373 times faster than the leading neural-field method RANF. We further validate the approach on HUTUBS, demonstrating its effectiveness across HRTF datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。