我本可以问那个：重构不可回答的问题
I Could've Asked That: Reformulating Unanswerable Questions
AI总结：
针对LLMs识别但不协助重构不可回答问题的局限，本文提出CouldAsk基准，评估发现GPT-4和Llama2-7B等模型重构成功率极低，主因是仅改写或重复原问题。
摘要：
在从不熟悉的文档中查找信息时，用户经常提出文档无法回答的问题。尽管现有的大型语言模型（LLMs）能够识别这些不可回答的问题，但它们并未协助用户重构问题，从而降低了其整体实用性。我们策划了CouldAsk，这是一个由现有和新数据集组成的评估基准，专门用于研究不可回答问题的重构。我们在CouldAsk上评估了最先进的开源和专有LLMs。结果表明这些模型在重构问题方面的能力有限。具体而言，GPT-4和Llama2-7B分别仅在26%和12%的情况下成功重构了问题。错误分析显示，62%的不成功重构源于模型仅对问题进行了改写甚至生成了完全相同的问题。我们公开发布了该基准和用于复现实验的代码。
英文摘要：
When seeking information from unfamiliar documents, users frequently pose questions that cannot be answered by the documents. While existing large language models (LLMs) identify these unanswerable questions, they do not assist users in reformulating their questions, thereby reducing their overall utility. We curate CouldAsk, an evaluation benchmark composed of existing and new datasets for document-grounded question answering, specifically designed to study reformulating unanswerable questions. We evaluate state-of-the-art open-source and proprietary LLMs on CouldAsk. The results demonstrate the limited capabilities of these models in reformulating questions. Specifically, GPT-4 and Llama2-7B successfully reformulate questions only 26% and 12% of the time, respectively. Error analysis shows that 62% of the unsuccessful reformulations stem from the models merely rephrasing the questions or even generating identical questions. We publicly release the benchmark and the code to reproduce the experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
