arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.10083cs.AI
收藏

超越图:大型语言模型能否理解超图?

Beyond Graphs: Can Large Language Models Comprehend Hypergraphs?

Tsinghua University(清华大学);Shanghai University(上海大学);Shenzhen University(深圳大学)

Yifan Feng, Chengwu Yang, Xingliang Hou, Shaoyi Du, Shihui Ying, Zongze Wu, Yue Gao

AI总结:

针对现有图基准忽略高阶关系的问题,提出首个超图理解基准LLM4Hypergraph,并设计专用提示框架与新技术Hyper-BAG和Hyper-COT,有效提升LLMs的高阶推理能力。

摘要:

现有的基准如NLGraph和GraphQA在评估LLMs时主要关注成对关系,忽略了现实世界数据中的高阶相关性。超图能够建模复杂的超越成对关系,提供了更稳健的框架,但在LLMs背景下仍被探索不足。为填补这一空白,我们引入了LLM4Hypergraph,这是首个综合性基准,包含21,500个问题,涵盖8个低阶、5个高阶和2个同构任务,利用来自引文网络和蛋白质结构的合成及真实超图。我们评估了包括GPT-4o在内的6个主流LLMs,证明了我们的基准在识别模型优缺点方面的有效性。我们的专用提示框架整合了7种超图语言,并引入了两种新技术Hyper-BAG和Hyper-COT,这些技术增强了高阶推理,在结构分类任务上平均提升4%(最高达9%)。这项工作为将超图计算能力集成到LLMs中建立了基础测试平台,推进了其理解能力。源代码位于https://github.com/iMoonLab/LLM4Hypergraph。

英文摘要:

Existing benchmarks like NLGraph and GraphQA evaluate LLMs on graphs by focusing mainly on pairwise relationships, overlooking the high-order correlations found in real-world data. Hypergraphs, which can model complex beyond-pairwise relationships, offer a more robust framework but are still underexplored in the context of LLMs. To address this gap, we introduce LLM4Hypergraph, the first comprehensive benchmark comprising 21,500 problems across eight low-order, five high-order, and two isomorphism tasks, utilizing both synthetic and real-world hypergraphs from citation networks and protein structures. We evaluate six prominent LLMs, including GPT-4o, demonstrating our benchmark's effectiveness in identifying model strengths and weaknesses. Our specialized prompting framework incorporates seven hypergraph languages and introduces two novel techniques, Hyper-BAG and Hyper-COT, which enhance high-order reasoning and achieve an average 4% (up to 9%) performance improvement on structure classification tasks. This work establishes a foundational testbed for integrating hypergraph computational capabilities into LLMs, advancing their comprehension. The source codes are at https://github.com/iMoonLab/LLM4Hypergraph.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑