大语言模型对含噪指令的鲁棒性
Resilience of Large Language Models for Noisy Instructions
AI总结：
本文研究大语言模型对ASR、OCR、语法、打字及干扰内容五类噪声指令的鲁棒性，发现模型整体性能显著下降，并评估了re-pass净化策略，指出纠正含噪指令尤其是开源LLMs仍面临重大挑战。
摘要：
作为快速发展的自然语言处理（NLP）领域，大语言模型（LLMs）已成为在各种任务中解释人类命令和生成文本的强大工具。然而，LLMs处理包含固有错误的文本的鲁棒性尚未得到充分探索，这些错误源于人类交互和协作系统。我们的研究调查了LLMs对五种常见干扰类型的鲁棒性，包括：1）ASR（自动语音识别）错误，2）OCR（光学字符识别）错误，3）语法错误，4）打字错误，以及5）干扰性内容。我们旨在通过故意将这些错误嵌入指令中来研究这些模型的反应。我们的研究结果表明，虽然一些LLMs对某些类型的噪声表现出一定程度的抵抗力，但它们的整体性能显著下降。这强调了进一步研究增强模型鲁棒性的重要性。针对观察到的性能下降，我们的研究还评估了一种“重新传递”（re-pass）策略，旨在LLMs处理指令之前净化指令中的噪声。我们的分析表明，纠正含噪指令，特别是对于开源LLMs，面临重大挑战。
英文摘要：
As the rapidly advancing domain of natural language processing (NLP), large language models (LLMs) have emerged as powerful tools for interpreting human commands and generating text across various tasks. Nonetheless, the resilience of LLMs to handle text containing inherent errors, stemming from human interactions and collaborative systems, has not been thoroughly explored. Our study investigates the resilience of LLMs against five common types of disruptions including 1) ASR (Automatic Speech Recognition) errors, 2) OCR (Optical Character Recognition) errors, 3) grammatical mistakes, 4) typographical errors, and 5) distractive content. We aim to investigate how these models react by deliberately embedding these errors into instructions. Our findings reveal that while some LLMs show a degree of resistance to certain types of noise, their overall performance significantly suffers. This emphasizes the importance of further investigation into enhancing model resilience. In response to the observed decline in performance, our study also evaluates a "re-pass" strategy, designed to purify the instructions of noise before the LLMs process them. Our analysis indicates that correcting noisy instructions, particularly for open-source LLMs, presents significant challenges.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
