TESLA：正弦可学习激活函数的泰勒展开
TESLA: Taylor Expansion of Sinusoidal Learnable Activations
浏览论文内容
中文总结 AI 辅助
针对标准神经网络难以解决的奇偶问题，提出TESLA激活函数，通过正弦余弦可学习组合实现多项式次数控制与高阶分量放大，在奇偶等任务上表现优异且可迁移至视觉任务。
中文摘要 AI 辅助
奇偶问题——判断二进制向量中1的数量是奇数还是偶数——对于标准神经网络来说仍然具有挑战性，这源于线性不可分性以及对全局交互的需求。我们提出TESLA，一种定义为正弦和余弦项可学习组合的激活函数，可实现对多项式次数的显式控制并选择性放大高阶分量。理论上，我们证明约束TESLA的系数可得到Lipschitz/拉德马赫复杂度界，并塑造训练动态以强调更高频结构。经验上，在输入长度n=32的奇偶问题上，TESLA使用10万个训练样本（约为2^32输入空间的0.002%）实现了强泛化，且在严重损坏下仍保持鲁棒性，在高达30%的标签噪声下仍保留高准确率。我们还在奇偶问题和Forrelation任务上将其与周期性和基于频率的基线（SIREN、SNAKE及傅里叶特征嵌入）进行比较。除合成结构外，TESLA在ImageNet-100上也表现出可比性能，表明激活级别的次数控制可迁移到更通用的视觉任务中。代码：this https URL
英文摘要
The parity problem--deciding whether the number of ones in a binary vector is odd or even--remains challenging for standard neural networks due to linear inseparability and the need for global interactions. We propose TESLA, an activation defined as a learnable combination of sine and cosine terms, enabling explicit control over polynomial degree and selective amplification of high-order components. Theoretically, we show that constraining TESLA's coefficients yields Lipschitz/Rademacher complexity bounds and shapes the training dynamics to emphasize higher-frequency structure. Empirically, on parity with input length n = 32, TESLA attains strong generalization with 100K training samples (approximately 0.002% of the 2^32 input space) and remains robust under heavy corruption, retaining high accuracy with up to 30% label noise. We also compare against periodic and frequency-based baselines (SIREN, SNAKE, and Fourier feature embeddings) on parity and Forrelation. Beyond synthetic structure, TESLA delivers comparable performance on ImageNet-100, indicating that activation-level degree control transfers to more general vision workloads. Code: https://github.com/KAU-QuantumAILab/TESLA
发表机构
- Korea Aerospace University（韩国航空大学）
机构由 AI 辅助整理，请以论文原文为准。
