Alpha-Alternator：利用Vendi分数动态适应序列中变化的噪声水平以提升鲁棒性和性能
The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance
AI总结：
本文提出α-Alternator生成模型，利用Vendi分数动态调整序列元素与潜在历史的影响权重，以应对变化噪声，提升轨迹预测、插补和预测性能，并在神经解码和时间序列基准上超越现有模型。
摘要：
当前最先进的动力学模型（如Mamba）假定给定序列的所有元素具有相同的噪声水平，这限制了它们在含噪时间数据上的性能。在本文中，我们引入了α-Alternator，一种用于时间依赖数据的新型生成模型，它能够动态适应序列中不同噪声水平所带来的复杂性。α-Alternator利用Vendi分数（VS），一种灵活的基于相似性的多样性度量，在每个时间步t调整序列元素在时间t的影响以及截至该时间步的动力学潜在表示对预测未来动力学的影响。该影响由一个参数捕获，该参数在给定数据集的所有序列中学习并共享。该参数的符号决定了影响的方向。负值表示数据集含噪，此时增加VS的序列元素被视为含噪，模型在处理该元素时更多地依赖潜在历史。相反，当参数为正时，增加VS的序列元素被视为信息丰富，α-Alternator在更新其预测的潜在动力学时更多地依赖这个新输入而非潜在历史。α-Alternator通过结合观测掩蔽和Alternator损失最小化进行训练。掩蔽模拟序列中变化的噪声水平，使模型对这些波动更加鲁棒，并提升其在轨迹预测、插补和预测方面的性能。我们的实验结果表明，α-Alternator在神经解码和时间序列预测基准上均优于Alternator和最先进的状态空间模型。
英文摘要：
Current state-of-the-art dynamical models, such as Mamba, assume the same level of noisiness for all elements of a given sequence, which limits their performance on noisy temporal data. In this paper, we introduce the $α$-Alternator, a novel generative model for time-dependent data that dynamically adapts to the complexity introduced by varying noise levels in sequences. The $α$-Alternator leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to adjust, at each time step $t$, the influence of the sequence element at time $t$ and the latent representation of the dynamics up to that time step on the predicted future dynamics. This influence is captured by a parameter that is learned and shared across all sequences in a given dataset. The sign of this parameter determines the direction of influence. A negative value indicates a noisy dataset, where a sequence element that increases the VS is considered noisy, and the model relies more on the latent history when processing that element. Conversely, when the parameter is positive, a sequence element that increases the VS is considered informative, and the $α$-Alternator relies more on this new input than on the latent history when updating its predicted latent dynamics. The $α$-Alternator is trained using a combination of observation masking and Alternator loss minimization. Masking simulates varying noise levels in sequences, enabling the model to be more robust to these fluctuations and improving its performance in trajectory prediction, imputation, and forecasting. Our experimental results demonstrate that the $α$-Alternator outperforms both Alternators and state-of-the-art state-space models across neural decoding and time-series forecasting benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
