RotRNN：利用旋转建模长序列
RotRNN: Modelling Long Sequences with Rotations
AI总结：
提出RotRNN线性循环模型，利用旋转矩阵简化初始化与归一化，在长序列基准上达到竞争性能。
摘要：
线性循环神经网络，如状态空间模型（SSMs）和线性循环单元（LRUs），最近在长序列建模基准上展示了最先进的性能。尽管取得了成功，但其经验性能尚未得到充分理解，并且它们带来了一些缺点，最显著的是复杂的初始化和归一化方案。在这项工作中，我们通过提出RotRNN——一种利用旋转矩阵便利性质的线性循环模型——来解决其中一些问题。我们展示了RotRNN提供了一个简单且高效的模型，具有稳健的归一化过程，以及一个忠实于其理论推导的实用实现。RotRNN还在多个长序列建模数据集上达到了与最先进的线性循环模型相竞争的性能。
英文摘要：
Linear recurrent neural networks, such as State Space Models (SSMs) and Linear Recurrent Units (LRUs), have recently shown state-of-the-art performance on long sequence modelling benchmarks. Despite their success, their empirical performance is not well understood and they come with a number of drawbacks, most notably their complex initialisation and normalisation schemes. In this work, we address some of these issues by proposing RotRNN -- a linear recurrent model which utilises the convenient properties of rotation matrices. We show that RotRNN provides a simple and efficient model with a robust normalisation procedure, and a practical implementation that remains faithful to its theoretical derivation. RotRNN also achieves competitive performance to state-of-the-art linear recurrent models on several long sequence modelling datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
