arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.07140cs.AI
收藏

图的描述顺序能否影响大语言模型解决图问题?

Can Graph Descriptive Order Affect Solving Graph Problems with LLMs?

Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of California, Merced(加州大学默塞德分校);SAIDS, University of Science and Technology of China(中国科学技术大学SAIDS);Tsinghua University(清华大学)

Yuyao Ge, Shenghua Liu, Baolong Bi, Yiwei Wang, Lingrui Mei, Wenjie Feng, Lizhe Chen, Xueqi Cheng

AI总结:

本研究首次系统分析图描述顺序对LLM解决图问题性能的影响,发现有序描述可显著提升模型对图结构的理解,且影响因任务而异。

摘要:

大语言模型(LLMs)在推理任务中取得了显著成功,包括数学推理和逻辑演绎。在这些推理任务中,图问题因其复杂性和独特的结构特征而脱颖而出,吸引了研究人员的广泛关注。以往的研究通过多种技术探索了LLMs的图推理能力,例如针对图结构的不同编码方法以及精心设计的提示的使用。然而,一个关键因素在很大程度上被忽视了:图描述呈现给模型的提示顺序。在本研究中,我们首次全面分析了图描述顺序如何影响LLM性能。具体而言,我们使用六个主流LLM,在六个图问题上全面评估了四种图描述顺序。结果显示:(1)有序的图描述显著提高了LLMs对图结构的理解;(2)LLMs对图描述顺序的鲁棒性在不同任务间存在差异;(3)图顺序对性能的影响与任务的内在特征密切相关。本研究为LLMs在解决图相关问题中的应用提供了关键进展,为未来通过策略性图描述排序优化模型性能的研究铺平了道路。

英文摘要:

Large language models (LLMs) have achieved significant success in reasoning tasks, including mathematical reasoning and logical deduction. Among these reasoning tasks, graph problems stand out due to their complexity and unique structural characteristics, attracting considerable attention from researchers. Previous studies have explored LLMs' graph reasoning abilities through various techniques, such as different encoding methods for graph structures and the use of carefully designed prompts. However, a critical factor has been mostly overlooked: the prompt sequential order in which graph descriptions are presented to the models. In this study, we present the first comprehensive analysis of how the order of graph descriptions impacts LLM performance. Specifically, we comprehensively evaluate four graph description orders across six graph problems using six mainstream LLMs. The results reveal that: (1) ordered graph descriptions significantly improve LLMs' comprehension of graph structures; (2) the robustness of LLMs to graph description order varies across different tasks; and (3) the impact of graph order on performance is closely related to the inherent characteristics of tasks. This study provides a critical advancement in the application of LLMs for solving graph-related problems, paving the way for future research to optimize model performance through strategic graph description ordering.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑