Hadamard表示：跨无模型强化学习的性能支撑
Hadamard Representation: Scaffolding Performance Across Model-free RL
AI总结：
提出Hadamard表示（HR），通过将标准隐藏层替换为两个独立参数化层的逐元素乘积，减少神经元休眠并增加有效秩，从而在多种强化学习算法和领域中一致提升性能。
摘要：
深度强化学习智能体在训练过程中逐渐失去表示能力：神经元变得休眠，从网络中移除活跃容量，有效秩崩溃，使存活的神经元冗余。现有的补救措施如周期性重置和特殊神经网络架构，大多局限于特定算法或领域。我们提出一个简单的架构修复，即Hadamard表示（HR），它将标准隐藏层替换为两个独立参数化层的逐元素乘积。HR通过两种互补机制运作。首先，它降低了神经元变得休眠的概率，这对于连续可微激活函数（如tanh）尤其有价值：与休眠的ReLU神经元（被有效剪枝）不同，饱和的tanh神经元通过将其输出权重转化为固定偏置而暗中破坏下游层。其次，独立于休眠，乘法结构捕获更丰富的特征交互，并在不拓宽层的情况下增加有效秩。我们在五种算法和三个领域上评估HR：基于像素的离散动作Atari上的DQN、PPO和PQN，基于状态连续控制上的SimbaV2，以及视觉连续控制上的MR.Q。HR在无需任何超参数调优的情况下，一致地优于强基线，并且其增益在参数匹配的更宽变体上仍然保持，排除了参数数量作为替代解释的可能性。
英文摘要：
Deep reinforcement learning agents progressively lose representational capacity during training: neurons become dormant, removing active capacity from the network, and effective rank collapses, leaving surviving neurons redundant. Existing remedies such as periodic resets, and special neural network architectures, are largely algorithm- or domain-specific. We propose a simple architectural fix, the Hadamard Representation (HR), which replaces a standard hidden layer with the element-wise product of two independently parameterized layers. HR operates through two complementary mechanisms. First, it reduces the probability of a neuron becoming dormant, which is particularly valuable for continuously differentiable activations such as tanh: unlike dormant ReLU neurons, which are effectively pruned, saturated tanh neurons silently corrupt downstream layers by turning their outgoing weights into fixed biases. Second, independently of dormancy, the multiplicative structure captures richer feature interactions and increases effective rank without widening the layer. We evaluate HR across five algorithms and three domains: DQN, PPO, and PQN on pixel-based discrete-action Atari, SimbaV2 on state-based continuous control, and MR.Q on visual continuous control. HR consistently improves performance over the strong baselines without any hyperparameter tuning, and gains persist against parameter-matched wider variants, ruling out parameter count as an alternative explanation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
