Neuronal diversity can improve machine learning for physics and beyond
North Carolina State University（北卡罗来纳州立大学）；The Jackson Laboratory for Genomic Medicine（杰克逊基因组医学实验室）；The College of Wooster（伍斯特学院）；Indian Institute of Science Education and Research Mohali（印度科学教育与研究学院莫哈利分校）
Anshul Choudhary, Anil Radhakrishnan, John F. Lindner, Sudeshna Sinha, William L. Ditto
Diversity conveys advantages in nature, yet homogeneous neurons typically comprise the layers of artificial neural networks. Here we construct neural networks from neurons that learn their own activation functions, quickly diversify, and subsequently outperform their homogeneous counterparts on image classification and nonlinear regression tasks. Sub-networks instantiate the neurons, which meta-learn especially efficient sets of nonlinear responses. Examples include conventional neural networks classifying digits and forecasting a van der Pol oscillator and physics-informed Hamiltonian neural networks learning Hénon-Heiles stellar orbits and the swing of a video recorded pendulum clock. Such \textit{learned diversity} provides examples of dynamical systems selecting diversity over uniformity and elucidates the role of diversity in natural and artificial systems.
One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networks
Chinese University of Hong Kong（香港中文大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Florida（佛罗里达大学）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
受生物神经系统中神经元多样性的启发，大量的研究提出设计新型的人工神经元，并将神经元多样性引入人工神经网络。最近提出的二次神经元，它取代了传统的神经元的内积运算与二次之一，取得了巨大的成功，在许多重要的任务。尽管二次神经元的结果很有希望，但仍然有一个未解决的问题：\textit{Is the superior performance of quadratic networks simply due to the increased parameters or due to the intrinsic expressive capability?}在没有澄清这个问题的情况下，二次网络的性能总是可疑的。此外，解决这个问题可以归结为寻找二次网络的杀手级应用。本文通过理论和实证研究，证明了二次网络具有参数有效性，从而证实了二次网络的优越性能是由于其内在的表达能力。这种内在的表达能力来自于二次神经元可以很容易地表示非线性相互作用，而传统的神经元很难。理论上，我们推导了二次网络在实空间和流形上的逼近效率。此外，从Barron空间的角度来看，我们证明了存在一个功能空间，其功能可以近似的二次网络的无量纲误差，但传统的网络的近似误差是依赖于尺寸。从经验上讲，对合成数据、经典基准测试和现实应用的实验结果表明，二次模型广泛享有参数效率，效率的提高取决于任务。
Beyond Word Error Rate: Auditing the Diversity Tax in Speech Recognition through Dataset Cartography
Department of Applied Mathematics and Computer Science, Technical University of Denmark（应用数学与计算机科学系，丹麦技术大学）；Department of Mathematical Sciences, University of Copenhagen（数学科学系，哥本哈根大学）
Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation
Nanjing University（南京大学）；Australia National University（澳大利亚国立大学）；Wuhan University（武汉大学）；National University of Singapore（新加坡国立大学）；University of Technology Sydney（技术科技大学）
Beyond Instance Consistency: Investigating View Diversity in Self-supervised Learning
Institute for Infocomm Research (I 2 R), A*STAR, Singapore（新加坡*STAR信息与通信研究所）；National University of Singapore, Singapore（新加坡国立大学）；Sichuan University, China（四川大学）；Southeast University, China（东南大学）；Shanghai Jiaotong University, China（上海交通大学）
Augmenting learning in neuro-embodied systems through neurobiological first principles
Newcastle University（纽卡斯尔大学）；University of Tokyo（东京大学）；University of Quebec in Trois-Rivieres（魁北克大学三河分校）；IT:U Interdisciplinary Transformation University Austria（奥地利跨学科转型大学）；Okinawa Institute of Science and Technology Graduate University（冲绳科学技术大学院大学）
Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization
South China University of Technology（华南理工大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Ocean University of China（中国海洋大学）；Center for AI Theoretical Foundation and Systems, Shenzhen Loop Area Institute（深圳环区研究所人工智能理论基础与系统中心）
Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding
CAS Key Laboratory of AI Security, Institute of Computing Technology, CAS（中国科学院人工智能安全重点实验室 中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）
