发表机构
Business School, University of Edinburgh（爱丁堡大学商学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
基于高斯互补一阶损失函数对GELU给出互补解释，将损失核算与前向信号传输分离，推广到阈值传输族，实验表明校准或学习的均匀阈值门有竞争力且能改进。
AI 中文摘要
高斯误差线性单元通常被视为输入相关随机伯努利门的预期输出。本文基于高斯互补一阶损失函数给出一种互补解释：GELU是具有高斯随机阈值的硬线性门预期盈余的信号传输项。此观点推广到包括ReLU、GELU、SiLU/Swish和硬Swish的阈值传输族，实验表明校准或学习的均匀阈值门有竞争力且能改进。
英文摘要
The Gaussian Error Linear Unit is usually motivated as the expected output of an input-dependent Bernoulli gate. This work gives an alternative interpretation: GELU is the expected output of a hard linear gate with a Gaussian random threshold. This view provides a generative interpretation for the Bernoulli gate: the gate opens once the input clears a latent Gaussian threshold. This interpretation stems from a decomposition based on well-known results in stochastic inventory theory and leads to a threshold-transmission family that includes ReLU, GELU, SiLU/Swish, and hard swish as special cases. By considering a latent uniform threshold, we recover a hard-swish-like piecewise-polynomial gate whose nonlinear transition is confined to a finite interval, yielding fixed- and learned-width variants. Controlled experiments on compact vision and language models show that calibrated or learned uniform-threshold gates are consistently competitive with GELU, ReLU, and SiLU/Swish, display architecture-dependent learned widths, and use the finite transition region nontrivially.
Comments18 pages, 8 figures, 8 tables
