越低越简单：简化分层递归模型
The Lower The Simpler: Simplifying Hierarchical Recurrent Models
摘要：
为了提高层次递归模型的训练效率而不影响其性能，我们提出了一种名为“越低越简单”的策略，即通过使下层比上层更简单来简化基线模型。我们执行这一策略，以简化两个典型的层次递归模型，即层次递归编码器-解码器（HRED）和R-NET，其基本构建块是GRU。具体来说，我们提出了标量门控单元（SGU），这是一个简化的GRU的变体，并使用它来取代在HRED和R-NET的中间层的GRU。此外，我们还使用固定大小的有序遗忘编码（FOFE），这是一种有效的编码方法，没有任何可训练的参数，以取代在HRED和R-NET的底层的GRU。实验结果表明，简化的HRED和简化的R-NET包含更少的可训练参数，消耗更少的训练时间，并实现略优于其基线模型的性能。
英文摘要：
To improve the training efficiency of hierarchical recurrent models without compromising their performance, we propose a strategy named as `the lower the simpler', which is to simplify the baseline models by making the lower layers simpler than the upper layers. We carry out this strategy to simplify two typical hierarchical recurrent models, namely Hierarchical Recurrent Encoder-Decoder (HRED) and R-NET, whose basic building block is GRU. Specifically, we propose Scalar Gated Unit (SGU), which is a simplified variant of GRU, and use it to replace the GRUs at the middle layers of HRED and R-NET. Besides, we also use Fixed-size Ordinally-Forgetting Encoding (FOFE), which is an efficient encoding method without any trainable parameter, to replace the GRUs at the bottom layers of HRED and R-NET. The experimental results show that the simplified HRED and the simplified R-NET contain significantly less trainable parameters, consume significantly less training time, and achieve slightly better performance than their baseline models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
