arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.07069cs.CLcs.AIcs.LG
收藏

上下文至关重要:面向科学应用的大型语言模型数据高效增强

Context Matters: Data-Efficient Augmentation of Large Language Models for Scientific Applications

University of Southern California(南加州大学);Information Sciences Institute, University of Southern California(南加州大学信息科学研究所)

Xiang Li, Haoran Tang, Siyu Chen, Ziwei Wang, Anurag Maravi, Marcin Abram

AI总结:

本研究针对大型语言模型在科学应用中易出错的问题,发现上下文相关性与答案质量呈非线性关系,并证明通过校准可自动化评分,实现LLM自我检查,同时提供了概念验证平台。

摘要:

在本文中,我们探讨了大型语言模型(LLMs)如GPT-4所固有的挑战,特别是它们在回答复杂问题时倾向于产生幻觉、逻辑错误和不正确结论的问题。LLMs以连贯且语义严谨的方式呈现错误答案的能力,进一步增加了检测事实不准确的难度。这一问题在需要专业知识的领域尤为突出。我们的工作深入研究了这些挑战,旨在增强对这些错误的理解并缓解它们,从而为提升LLMs在科学及其他专业领域的准确性和可靠性做出贡献。我们的研究结果揭示了上下文相关性与答案测量质量之间的非线性关系。此外,我们证明,通过正确的校准,可以自动化评分过程——这一发现表明,至少在一定程度上,LLMs可以被用于自我检查其自身表现的质量。最后,我们描述了一个实验平台,该平台可视为本文所述技术的概念验证。

英文摘要:

In this paper, we explore the challenges inherent to Large Language Models (LLMs) like GPT-4, particularly their propensity for hallucinations, logic mistakes, and incorrect conclusions when tasked with answering complex questions. The capacity of LLMs to present erroneous answers in a coherent and semantically rigorous manner further complicates the detection of factual inaccuracies. This issue is especially pronounced in fields that require specialized expertise. Our work delves into these challenges, aiming to enhance the understanding and mitigation of such errors, thereby contributing to the improvement of LLM accuracy and reliability in scientific and other specialized domains. Our findings reveal a non-linear relationship between the context's relevancy and the answers' measured quality. In addition, we demonstrate that with the correct calibration, it is possible to automate the grading procedure -- a finding suggesting that, at least to some degree, the LLMs can be used to self-examine the quality of their own performance. Finally, we describe an experimental platform that can be seen as a proof-of-concept of the techniques described in this work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑