arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1808.08149cs.CLcs.LGstat.ML
收藏

从随机到监督:一种融合全局信息的新型辍学机制

From Random to Supervised: A Novel Dropout Mechanism Integrated with Global Information

Beijing University of Post and Telecommunication(北京邮电大学);Beijing Normal University(北京师范大学)

Hengru Xu, Shen Li, Renfen Hu, Si Li, Sheng Gao

摘要:

Dropout用于通过在训练期间从神经网络随机丢弃单元来避免过拟合。受dropout方法的启发,提出了一种结合全局信息的dropout方法GI-Dropout,用于改进神经网络的文本分类。与传统的dropout方法不同,在dropout方法中,单元根据相同的概率随机丢弃,我们的目标是使用基于数据集全局信息的显式指令来指导训练过程。使用GI-Dropout,模型应该更加关注不明显的特征或模式。实验结果表明,辍学的有效性与全球信息的七个文本分类任务,包括情感分析和主题分类。

英文摘要:

Dropout is used to avoid overfitting by randomly dropping units from the neural networks during training. Inspired by dropout, this paper presents GI-Dropout, a novel dropout method integrating with global information to improve neural networks for text classification. Unlike the traditional dropout method in which the units are dropped randomly according to the same probability, we aim to use explicit instructions based on global information of the dataset to guide the training process. With GI-Dropout, the model is supposed to pay more attention to inapparent features or patterns. Experiments demonstrate the effectiveness of the dropout with global information on seven text classification tasks, including sentiment analysis and topic classification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑