Long-context capabilities are essential for large language models (LLMs) to tackle complex and long-input tasks. Despite numerous efforts made to optimize LLMs for long contexts, challenges persist in robustly processing long inputs. In this paper, we introduce GraphReader, a graph-based agent system designed to handle long texts by structuring them into a graph and employing an agent to explore this graph autonomously. Upon receiving a question, the agent first undertakes a step-by-step analysis and devises a rational plan. It then invokes a set of predefined functions to read node content and neighbors, facilitating a coarse-to-fine exploration of the graph. Throughout the exploration, the agent continuously records new insights and reflects on current circumstances to optimize the process until it has gathered sufficient information to generate an answer. Experimental results on the LV-Eval dataset reveal that GraphReader, using a 4k context window, consistently outperforms GPT-4-128k across context lengths from 16k to 256k by a large margin. Additionally, our approach demonstrates superior performance on four challenging single-hop and multi-hop benchmarks.
Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);School of Information, Renmin University of China(中国人民大学信息学院);Baichuan Inc(百川科技)
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models
Beijing University of Posts and Telecommunications(北京邮电大学);The Hong Kong University of Science and Technology(香港科技大学);Beihang University(北京航空航天大学);University of the Chinese Academy of Sciences(中国科学院大学);ETH Zürich(苏黎世联邦理工学院);The Hong Kong Polytechnic University(香港理工大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shanghai AI Lab(上海人工智能实验室)
Eliciting Causal Abilities in Large Language Models for Reasoning Tasks
BNU-HKBU United International College(北师港浸大);Hong Kong Baptist University(香港浸会大学);Beijing Normal University at Zhuhai(北京师范大学珠海校区);The Chinese University of Hong Kong(香港中文大学)
A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning
Institute for Artificial Intelligence, Tsinghua University(清华大学人工智能研究院);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Tencent AI Lab(腾讯AI实验室)
GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents
Olin College of Engineering(奥林工程学院);University of Texas at Dallas(德克萨斯大学达拉斯分校);University of Oxford(牛津大学);University College London(伦敦大学学院);ShanghaiTech University(上海科技大学);Columbia University(哥伦比亚大学);University of Manchester(曼彻斯特大学)
Self-evolving Agents with reflective and memory-augmented abilities
East China Jiao Tong University(东中国交通大学);University of Minnesota - Twin Cities(明尼苏达大学-双城分校);Emory University(埃默里大学);University of Electronic Science and Technology of China(电子科技大学);Guangdong University of Technology(广东工业大学);Amazon(亚马逊);Washington University(华盛顿大学);Xiamen University(厦门大学);University of Bristol(布里斯托尔大学);University of Toronto(多伦多大学)
Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences(中国科学院大学);Xiaohongshu Inc(小红书公司)
Self-Evolving Critique Abilities in Large Language Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院);Shenzhen International Center for Industrial and Applied Mathematics(深圳国际工业与应用数学中心);Qwen Team, Alibaba Inc.(阿里云通义团队)
AI Models Can Predict and Collaboratively Modulate Human Memory Search
Champalimaud Research(尚帕利莫德研究院);Champalimaud Centre for Restorative Neurotechnology(尚帕利莫德修复神经技术中心);New York University Tandon(纽约大学坦登工程学院);Indiana University(印第安纳大学)
AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models
Alibaba Group(阿里巴巴集团);School of Computer Science, Fudan University(复旦大学计算机科学技术学院);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学上海市智能信息处理重点实验室)
Crystal: Illuminating LLM Abilities on Language and Code
Carnegie Mellon University(卡内基梅隆大学);Cerebras Systems(赛布拉斯系统公司);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?
The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));National University of Singapore(新加坡国立大学)
ICLEval: Evaluating In-Context Learning Ability of Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Huawei Poisson Lab(华为泊松实验室);School of Information, Renmin University of China(中国人民大学信息学院)
Evaluating LLM Abilities to Understand Tabular Electronic Health Records: A Comprehensive Study of Patient Data Extraction and Retrieval
Université Paul Sabatier(保罗·萨巴蒂埃大学);IRIT(信息论与电信研究所);Centre Hospitalier Universitaire de Toulouse(图卢兹大学医院中心);INSERM(法国国家健康与医学研究院);Université de Toulouse(图卢兹大学);Faculté de Médecine(医学院)
Reasoning Can Hurt the Inductive Abilities of Large Language Models
School of Information Sciences University of Illinois at Urbana-Champaign(信息科学学院伊利诺伊大学厄巴纳-香槟分校);Computer Science and Engineering HKUST(计算机科学与工程香港科技大学);Research Scientist, Intel Labs(英特尔实验室研究员);School of Information Sciences University of Illinois Urbana-Champaign(信息科学学院伊利诺伊大学厄巴纳-香槟分校)
NUMINA: A Natural Understanding Benchmark for Multi-dimensional Intelligence and Numerical Reasoning Abilities
School of Advanced Technology, Xi’an Jiaotong-Liverpool University(西安交通大学利物浦大学先进技术学院);Department of Computer Science, University of Liverpool(利物浦大学计算机科学系);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Institute of Deep Perception Technology, JITRI(深度感知技术研究所)
Exploring the Multilingual NLG Evaluation Abilities of LLM-Based Evaluators
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);School of Data Science and Intelligent Media, Communication University of China(中国传媒大学数据科学与智能媒体学院)
TUMS: Enhancing Tool-use Abilities of LLMs with Multi-structure Handlers
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Nanjing Artificial Intelligence Research of IA(南京人工智能研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Nanjing University of Information Science & Technology(南京信息职业技术学院)
UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset
Tsinghua University(清华大学);Beijing Language and Culture University(北京语言大学);Northeastern University(东北大学);Beijing University of Posts and Telecommunications(北京邮电大学)