Though current long-context large language models (LLMs) have demonstrated impressive capacities in answering user questions based on extensive text, the lack of citations in their responses makes user verification difficult, leading to concerns about their trustworthiness due to their potential hallucinations. In this work, we aim to enable long-context LLMs to generate responses with fine-grained sentence-level citations, improving their faithfulness and verifiability. We first introduce LongBench-Cite, an automated benchmark for assessing current LLMs' performance in Long-Context Question Answering with Citations (LQAC), revealing considerable room for improvement. To this end, we propose CoF (Coarse to Fine), a novel pipeline that utilizes off-the-shelf LLMs to automatically generate long-context QA instances with precise sentence-level citations, and leverage this pipeline to construct LongCite-45k, a large-scale SFT dataset for LQAC. Finally, we train LongCite-8B and LongCite-9B using the LongCite-45k dataset, successfully enabling their generation of accurate responses and fine-grained sentence-level citations in a single output. The evaluation results on LongBench-Cite show that our trained models achieve state-of-the-art citation quality, surpassing advanced proprietary models including GPT-4o.
State Key Lab of AI Safety, Institute of Computing Technology, CAS（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；Key Lab of AI Safety, Chinese Academy of Sciences（人工智能安全重点实验室，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
GenProve: Learning to Generate Text with Fine-Grained Provenance
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)（计算能力网络与信息安全教育部重点实验室，山东省计算中心（国家超级计算济南中心），齐鲁工业大学（山东省科学院））；Shenyang Institute of Computing Technology, Chinese Academy of Sciences（中国科学院沈阳计算技术研究所）；MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统实验室）；University of Chinese Academy of Sciences（中国科学院大学）
When Large Language Models Meet Citation: A Survey
Macquarie University（麦考瑞大学）；Wuhan University（武汉大学）；Nanyang Technological University（南洋理工大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；University of New South Wales（新南威尔士大学）；The University of Adelaide（阿德莱德大学）
Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study
Department of Data Science & AI, Monash University（数据科学与人工智能系，莫纳什大学）；Faculty of Law, Monash University（法学院，莫纳什大学）
Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University（上海数据科学 key laboratory，计算机科学学院，复旦大学）；School of Data Science, Fudan University（数据科学学院，复旦大学）；AntGroup（蚂蚁集团）
Validate Your Authority: Benchmarking LLMs on Multi-Label Precedent Treatment Classification
Department of Information Science and Technology（信息科学与技术系）；College of Emergency Preparedness, Homeland Security, and Cybersecurity（应急准备、国土安全与网络安全学院）；University at Albany, SUNY（萨利纳大学）
Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation
Center for Language and Cognition (CLCG), University of Groningen（格罗宁根大学语言与认知中心（CLCG））；Institute for Logic, Language and Computation (ILLC), University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所（ILLC））
Enhancing Academic Paper Recommendations Using Fine-Grained Knowledge Entities and Multifaceted Document Embeddings
Department of Information Management（信息管理系）；Nanjing University of Science & Technology（南京理工大学）；School of Computer Engineering（计算机工程学院）；Jiangsu University of Technology（江苏科技大学）；School of Information Management（信息管理学院）；Central China Normal University（中央财经大学）
