从表层模式到语义理解：在对比集上微调语言模型
From Superficial Patterns to Semantic Understanding: Fine-Tuning Language Models on Contrast Sets
AI总结：
本文研究NLI模型依赖表层模式、在对比集上泛化差的问题，通过在训练中加入少量复杂对比集进行微调，使ELECTRA-small在对比集上的准确率由75%恢复至接近90%。
摘要：
大规模预训练语言模型已在自然语言推理（NLI）任务的标准数据集上展现出高性能。遗憾的是，这些评估可能具有误导性，因为尽管模型在分布内数据上表现良好，但在对比集等分布外测试集上表现不佳。对比集由经过扰动的数据实例构成，这些实例对输入作出非常微小但有意义的改变，从而改变黄金标签，揭示模型如何学习训练数据中的表层模式，而不是学习更复杂的语言细微差别。例如，ELECTRA-small语言模型在SNLI数据集上达到接近90%的准确率，但在分布外对比集上测试时降至75%。本研究开展的研究探讨了如何通过在训练期间让语言模型接触少量更复杂的对比集来提升其鲁棒性，以帮助其更好地学习语言模式。通过这种方法，该模型恢复了性能，并在对比集上达到接近90%的准确率，凸显了多样化且具有挑战性的训练数据的重要性。
英文摘要：
Large-scale pre-trained language models have demonstrated high performance on standard datasets for natural language inference (NLI) tasks. Unfortunately, these evaluations can be misleading, as although the models can perform well on in-distribution data, they perform poorly on out-of-distribution test sets, such as contrast sets. Contrast sets consist of perturbed instances of data that have very minor, but meaningful, changes to the input that alter the gold label, revealing how models can learn superficial patterns in the training data rather than learning more sophisticated language nuances. As an example, the ELECTRA-small language model achieves nearly 90% accuracy on an SNLI dataset but drops to 75% when tested on an out-of-distribution contrast set. The research carried out in this study explores how the robustness of a language model can be improved by exposing it to small amounts of more complex contrast sets during training to help it better learn language patterns. With this approach, the model recovers performance and achieves nearly 90% accuracy on contrast sets, highlighting the importance of diverse and challenging training data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
