多速率专家混合模型加速液态神经网络训练
Multi-Rate Mixture of Experts for Accelerating Liquid Neural Network Training
AI总结：
提出多速率专家混合框架，结合液态神经网络的多尺度动态与注意力机制，提升多变量时间序列建模的准确性和效率。
摘要：
多变量时间序列数据通常表现出复杂的时间依赖、不规则采样和跨多个时间尺度的异质动态，使得精确序列建模特别具有挑战性。传统的循环神经网络（RNN），如长短期记忆网络（LSTM），在离散时间下运行，可能难以有效捕捉连续和不规则的时间行为。液态神经网络（LNN）通过连续时间动态解决了其中一些限制，但标准LNN架构通常依赖单一动力系统，限制了其建模异质时间模式的能力。为了解决这些挑战，我们提出了一个基于液态神经网络的多速率专家混合（MR-MoE）框架。在所提出的架构中，多个基于LNN的专家以不同的时间尺度运行，使模型能够明确分离快速变化的动态和缓慢演变的时间趋势。门控网络进一步实现了基于输入条件的自适应专家专业化。此外，我们结合了特征级和时间注意力机制，以提高鲁棒性、可解释性和长程依赖建模能力。特征级注意力抑制噪声或无关变量，而时间注意力则选择性地关注信息丰富的历史状态。我们在一个复杂的多变量时间序列预测任务上评估了所提出的框架，并与强基线模型（包括LSTM、单体LNN和标准MoE模型）进行了比较。实验结果表明，所提出的MR-MoE框架在保持良好计算效率的同时，持续实现了改进的AUROC和AUPRC性能。这些结果突显了结合连续时间动态、多尺度专家分解和自适应注意力机制对时间序列建模的有效性。
英文摘要：
Multivariate time-series data often exhibit complex temporal dependencies, irregular sampling, and heterogeneous dynamics across multiple time scales, making accurate sequence modeling particularly challenging. Traditional recurrent neural networks (RNNs), such as Long Short-Term Memory (LSTM) networks, operate in discrete time and may struggle to effectively capture continuous and irregular temporal behaviors. Liquid Neural Networks (LNNs) address some of these limitations through continuous-time dynamics, but standard LNN architectures typically rely on a single dynamical system, limiting their ability to model heterogeneous temporal patterns. To address these challenges, we propose a Multi-Rate Mixture-of-Experts (MR-MoE) framework built on top of Liquid Neural Networks. In the proposed architecture, multiple LNN-based experts operate at distinct time scales, enabling the model to explicitly separate fast-changing dynamics from slow-evolving temporal trends. A gating network further enables adaptive expert specialization based on input conditions. In addition, we incorporate both feature-level and temporal attention mechanisms to improve robustness, interpretability, and long-range dependency modeling. Feature-level attention suppresses noisy or irrelevant variables, while temporal attention selectively focuses on informative historical states. We evaluate the proposed framework on a complex multivariate time-series prediction task and compare it against strong baselines, including LSTM, monolithic LNN, and standard MoE models. Experimental results demonstrate that the proposed MR-MoE framework consistently achieves improved AUROC and AUPRC performance while maintaining favorable computational efficiency. These results highlight the effectiveness of combining continuous-time dynamics, multi-scale expert decomposition, and adaptive attention mechanisms for time-series modeling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
