Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；National University of Singapore（新加坡国立大学）
Making the content generated by Large Language Model (LLM), accurate, credible and traceable is crucial, especially in complex knowledge-intensive tasks that require multi-step reasoning and each step needs knowledge to solve. Retrieval-augmented generation is good potential to solve this problem. However, where and how to introduce Information Retrieval (IR) to LLM is a big challenge. Previous work has the problems that wrong knowledge retrieved by IR misleads the LLM and interaction between IR and LLM breaks the reasoning chain of LLM. This paper proposes a novel framework named \textbf{Search-in-the-Chain} (SearChain) for the interaction between LLM and IR to solve the challenges. First, LLM generates the reasoning chain named Chain-of-Query (CoQ) where each node consists of an IR-oriented query-answer pair. Second, IR verifies the answer of each node of CoQ. It corrects the answer that is not consistent with the retrieved information when IR gives high confidence, which improves the credibility. Third, LLM can indicate its missing knowledge in CoQ and rely on IR to provide this knowledge to LLM. These operations improve the accuracy in terms of reasoning and knowledge. Finally, SearChain generates the reasoning process and marks references to supporting documents for each reasoning step, which improves traceability. Interaction with IR in SearChain forms a novel reasoning path based on a tree, which enables LLM to dynamically modify the direction of reasoning. Experiments show that SearChain outperforms state-of-the-art baselines on complex knowledge-intensive tasks including multi-hop Q\&A, slot filling, fact checking, and long-form Q\&A.
Tool Calling: Enhancing Medication Consultation via Retrieval-Augmented Large Language Models
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；East China University of Science and Technology（华东理工大学）；SenseTime Research（商汤科技研究院）
Enhancing NLP Robustness and Generalization through LLM-Generated Contrast Sets: A Scalable Framework for Systematic Evaluation and Adversarial Training
College of Natural Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校自然科学学院）
Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting
Institute of Artificial Intelligence and Future Networks, Beijing Normal University（北京师范大学人工智能与未来网络研究院）；Hong Kong Baptist University（香港浸会大学）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）；College of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
Expanding before Inferring: Enhancing Factuality in Large Language Models through Premature Layers Interpolation
Sun Yat-Sen University（中山大学）；Shenzhen MSU-BIT University（深圳MSU-BIT大学）；The Hong Kong Polytechnic University（香港理工大学）；UNSW, Sydney, NSW 2052, Australia（新南威尔士大学）；Shenzhen Key Laboratory for High Performance Data Mining, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳高性能数据挖掘重点实验室，深圳先进技术研究院，中国科学院）
Gradual Vigilance and Interval Communication: Enhancing Value Alignment in Multi-Agent Debates
National Engineering Research Center of Educational Big Data, Central China Normal University（华中师范大学教育大数据国家工程研究中心）；Faculty of Artificial Intelligence in Education, Central China Normal University（华中师范大学人工智能教育学部）
KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；The University of Osaka（大阪大学）
GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
Southern University of Science and Technology（南方科技大学）；Alibaba Group（阿里巴巴集团）；Shanghai University of Finance and Economics（上海财经大学）；Tsinghua University（清华大学）；Xiamen University（厦门大学）
Beyond Elicitation: Provision-based Prompt Optimization for Knowledge-Intensive Tasks
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学与工程学院）；School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（上海交通大学自动化与智能感知学院）；National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所人机混合增强智能全国重点实验室）
