arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.11541cs.CLcs.AI
收藏

赋能大型语言模型在工业领域特定问答中表现更佳

Empower Large Language Model to Perform Better on Industrial Domain-Specific Question Answering

Microsoft(微软);The Chinese University of Hong Kong(香港中文大学)

Fangkai Yang, Pu Zhao, Zezhong Wang, Lu Wang, Jue Zhang, Mohit Garg, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

AI总结:

本文提出MSQA基准数据集和一种新的模型交互范式,以增强大型语言模型在工业领域特定问答中的性能,实验证明该方法优于常用的检索增强方法。

摘要:

大型语言模型(LLM)在开放域任务中已获得广泛关注并取得了显著成果,但由于缺乏特定领域知识,其在真实工业领域特定场景中的表现平平。这一问题引起了广泛关注,但相关基准测试却很少。本文提供了一个名为MSQA的基准问答(QA)数据集,围绕微软产品及客户遇到的信息技术问题构建。该数据集包含行业云特定问答知识,这是通用大型语言模型未广泛覆盖的领域,因此非常适合评估旨在增强大型语言模型领域特定能力的方法。此外,我们提出了一种新的模型交互范式,能够赋能大型语言模型在其不擅长的领域特定任务上取得更佳性能。大量实验表明,采用我们方法的方式优于常用的带检索的大型语言模型方法。我们在https://aka.ms/Microsoft_QA提供了源代码和示例数据。

英文摘要:

Large Language Model (LLM) has gained popularity and achieved remarkable results in open-domain tasks, but its performance in real industrial domain-specific scenarios is average due to its lack of specific domain knowledge. This issue has attracted widespread attention, but there are few relevant benchmarks available. In this paper, we provide a benchmark Question Answering (QA) dataset named MSQA, centered around Microsoft products and IT technical problems encountered by customers. This dataset contains industry cloud-specific QA knowledge, an area not extensively covered in general LLMs, making it well-suited for evaluating methods aiming to enhance LLMs' domain-specific capabilities. In addition, we propose a new model interaction paradigm that can empower LLM to achieve better performance on domain-specific tasks where it is not proficient. Extensive experiments demonstrate that the approach following our method outperforms the commonly used LLM with retrieval methods. We make our source code and sample data available at: https://aka.ms/Microsoft_QA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑