ChatQA：在对话式问答与RAG任务上超越GPT-4
ChatQA: Surpassing GPT-4 on Conversational QA and RAG
AI总结：
本研究推出ChatQA模型系列，通过两阶段指令微调与对话式稠密检索器，在自研ChatRAG Bench基准上超越GPT-4系列模型，并开源相关资源推动领域发展。
摘要：
本研究推出ChatQA模型系列，该系列在检索增强生成（retrieval-augmented generation，RAG）和对话式问答（question answering，QA）任务上表现优于GPT-4。为提升生成质量，我们提出一种两阶段指令微调方法，可显著提高RAG性能。为实现高效检索，我们引入一款针对对话式QA优化的稠密检索器，其效果与现有最优查询重写模型相当，同时大幅降低部署成本。我们还提出ChatRAG Bench基准，包含10个数据集，覆盖RAG、表格相关问答、算术计算以及涉及不可回答问题的场景的全面评估。我们的ChatQA-1.0-70B（得分54.14）基于比GPT-4更弱的基础模型Llama2构建，在不依赖OpenAI GPT模型生成的任何合成数据的情况下，在ChatRAG Bench上的表现略优于GPT-4-0613（得分53.90）和GPT-4-Turbo-2024-04-09（得分54.03）。值得注意的是，Llama3-ChatQA-1.5-70B模型的准确率超过GPT-4-Turbo-2024-04-09，提升幅度达4.4%。为推动该领域研究，我们向社区开源了模型权重、指令微调数据、ChatRAG Bench以及检索器，访问地址为：https://chatqa-project.github.io/。
英文摘要：
In this work, we introduce ChatQA, a suite of models that outperform GPT-4 on retrieval-augmented generation (RAG) and conversational question answering (QA). To enhance generation, we propose a two-stage instruction tuning method that significantly boosts the performance of RAG. For effective retrieval, we introduce a dense retriever optimized for conversational QA, which yields results comparable to the alternative state-of-the-art query rewriting models, while substantially reducing deployment costs. We also present the ChatRAG Bench, which encompasses ten datasets covering comprehensive evaluations on RAG, table-related QA, arithmetic calculations, and scenarios involving unanswerable questions. Our ChatQA-1.0-70B (score: 54.14), built on Llama2, a weaker foundation model than GPT-4, can slightly outperform GPT-4-0613 (score: 53.90) and GPT-4-Turbo-2024-04-09 (score: 54.03) on the ChatRAG Bench, without relying on any synthetic data from OpenAI GPT models. Notably, the Llama3-ChatQA-1.5-70B model surpasses the accuracy of GPT-4-Turbo-2024-04-09, achieving a 4.4% improvement. To advance research in this field, we open-sourced the model weights, instruction tuning data, ChatRAG Bench, and retriever for the community: https://chatqa-project.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
