评估语言模型在多样化挑战集上的并发鲁棒性
Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets
AI总结：
本文提出多种微调策略和思维链提示方法，系统评估并提升语言模型在多样化输入扰动下的并发鲁棒性，并在Tabular-NLI任务上验证其有效性。
摘要：
语言模型因其黑箱特性，常常产生幻觉并对输入扰动表现出敏感性，这引发了对其可信度的担忧。为了增强可信度，必须全面理解模型的失效模式，并制定有效的策略来提升其性能。在本研究中，我们提出了一种方法，旨在考察输入扰动如何影响不同规模的语言模型，包括预训练模型和大型语言模型（LLMs）。通过利用微调，我们增强了模型对输入扰动的鲁棒性。此外，我们探究了暴露于一种扰动是否会增强或削弱模型在面对其他扰动时的表现。为了解决对多种扰动的鲁棒性问题，我们提出了三种不同的微调策略。进一步地，我们通过利用带有示例增强的思维链（CoT）提示方法，将我们的方法扩展到大型语言模型（LLMs）。我们采用Tabular-NLI任务来展示我们提出的策略如何熟练地训练出一个鲁棒模型，使其能够应对多样化的扰动，同时在原始数据集上保持准确性。
英文摘要：
Language models, characterized by their black-box nature, often hallucinate and display sensitivity to input perturbations, causing concerns about trust. To enhance trust, it is imperative to gain a comprehensive understanding of the model's failure modes and develop effective strategies to improve their performance. In this study, we introduce a methodology designed to examine how input perturbations affect language models across various scales, including pre-trained models and large language models (LLMs). Utilizing fine-tuning, we enhance the model's robustness to input perturbations. Additionally, we investigate whether exposure to one perturbation enhances or diminishes the model's performance with respect to other perturbations. To address robustness against multiple perturbations, we present three distinct fine-tuning strategies. Furthermore, we broaden the scope of our methodology to encompass large language models (LLMs) by leveraging a chain of thought (CoT) prompting approach augmented with exemplars. We employ the Tabular-NLI task to showcase how our proposed strategies adeptly train a robust model, enabling it to address diverse perturbations while maintaining accuracy on the original dataset. https://msin-infotabs.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
