arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.08240cs.AIcs.SD
收藏

PTS-SNN:一种基于提示的时序脉冲神经网络用于高效的语音情感识别

PTS-SNN: A Prompt-Tuned Temporal Shift Spiking Neural Networks for Efficient Speech Emotion Recognition

College of Artificial Intelligence, Southwest University, Chongqing, 400715, China(人工智能学院,西南大学,重庆,400715,中国);Chongqing Key Laboratory of Brain Inspired Computing and Intelligent Chips, Chongqing, 400715, China(重庆脑启发计算与智能芯片重点实验室,重庆,400715,中国)

Xun Su, Huamin Wang, Qi Zhang

AI总结:

PTS-SNN通过时序移位脉冲编码器和上下文感知膜电位校准策略,实现高效的语音情感识别,准确率与人工神经网络相当,能耗显著降低。

摘要:

语音情感识别(SER)被广泛应用于人机交互,但传统模型的高计算成本阻碍了其在资源受限的边缘设备上的实施。脉冲神经网络(SNNs)因其事件驱动的性质提供了一种节能的替代方案;然而,将其与连续自监督学习(SSL)表示整合受到分布不匹配的根本挑战,其中高动态范围的嵌入会降低阈值神经元的信息编码能力。为了解决这个问题,我们提出了提示调整的脉冲神经网络(PTS-SNN),这是一种参数高效的神经形态适应框架,将冻结的SSL主干与脉冲动态对齐。具体来说,我们引入了一个时序移位脉冲编码器,通过无参数的通道移位捕捉局部时间依赖性,建立稳定的特征基础。为了弥合领域差距,我们设计了一种上下文感知膜电位校准策略。该机制利用一个脉冲稀疏线性注意力模块,将全局语义上下文聚合到可学习的软提示中,这些提示动态调节参数泄漏积分-放电(PLIF)神经元的偏置电压。这种调节有效将异构输入分布集中到响应放电范围内,缓解功能沉默或饱和。在五个多语言数据集(如IEMOCAP、CASIA、EMODB)上的广泛实验表明,PTS-SNN在IEMOCAP上实现了73.34%的准确率,与竞争性人工神经网络(ANNs)相当,同时仅需1.19M个可训练参数和0.35 mJ每样本的推理能耗。

英文摘要:

Speech Emotion Recognition (SER) is widely deployed in Human-Computer Interaction, yet the high computational cost of conventional models hinders their implementation on resource-constrained edge devices. Spiking Neural Networks (SNNs) offer an energy-efficient alternative due to their event-driven nature; however, their integration with continuous Self-Supervised Learning (SSL) representations is fundamentally challenged by distribution mismatch, where high-dynamic-range embeddings degrade the information coding capacity of threshold-based neurons. To resolve this, we propose Prompt-Tuned Spiking Neural Networks (PTS-SNN), a parameter-efficient neuromorphic adaptation framework that aligns frozen SSL backbones with spiking dynamics. Specifically, we introduce a Temporal Shift Spiking Encoder to capture local temporal dependencies via parameter-free channel shifts, establishing a stable feature basis. To bridge the domain gap, we devise a Context-Aware Membrane Potential Calibration strategy. This mechanism leverages a Spiking Sparse Linear Attention module to aggregate global semantic context into learnable soft prompts, which dynamically regulate the bias voltages of Parametric Leaky Integrate-and-Fire (PLIF) neurons. This regulation effectively centers the heterogeneous input distribution within the responsive firing range, mitigating functional silence or saturation. Extensive experiments on five multilingual datasets (e.g., IEMOCAP, CASIA, EMODB) demonstrate that PTS-SNN achieves 73.34\% accuracy on IEMOCAP, comparable to competitive Artificial Neural Networks (ANNs), while requiring only 1.19M trainable parameters and 0.35 mJ inference energy per sample.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑