语言大模型 / LLM - arXivDaily 专题

2510.25947 2025-10-31 cs.CL cs.AI cs.LG 90%

Revisiting Multilingual Data Mixtures in Language Model Pretraining

Negar Foroutan, Paul Teiletche, Ayush Kumar Tarun, Antoine Bosselut

机构 * EPFL（瑞士联邦理工学院）

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Under Review

2505.23811 2025-10-27 cs.CL cs.AI cs.LG 90%

LayerIF: Estimating Layer Quality for Large Language Models using Influence Functions

Hadi Askari, Shivanshu Gupta, Fei Wang, Anshuman Chhabra, Muhao Chen

机构 * University of California, Davis（加州大学戴维斯分校）； University of California, Irvine（加州大学伊市分校）； University of Southern California（南加州大学）； University of South Florida（佛罗里达州立大学）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Neurips 2025

2506.10946 2025-10-23 cs.LG cs.AI cs.CL 90%

GUARD: Guided Unlearning and Retention via Data Attribution for Large Language Models

Peizhi Niu, Evelyn Ma, Huiting Zhou, Duo Zhou, Huan Zhang, S. Rasoul Etesami, Olgica Milenkovic

机构 * Department of Electrical & Computer Engineering（电气与计算机工程系）； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2509.20149 2025-10-21 cs.SE 90%

Synergistic Enhancement of Requirement-to-Code Traceability: A Framework Combining Large Language Model based Data Augmentation and an Advanced Encoder

Jianzhang Zhang, Jialong Zhou, Nan Niu, Jinping Hua, Chuang Liu

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

2510.09051 2025-10-13 cs.CL cs.AI cs.LG 90%

Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation

Muhammad Ali Shafique, Kanwal Mehreen, Muhammad Arham, Maaz Amjad, Sabur Butt, Hamza Farooq

机构 * University of British Columbia（不列颠哥伦比亚大学）； Texas Tech University（德克萨斯技术大学）； Institute for the Future of Education, Tecnológico de Monterrey（教育未来研究所，墨西哥蒙特雷技术学院）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the EMNLP 2025 Workshop on Multilingual Representation Learning (MRL)

2501.02441 2025-10-07 stat.ML cs.AI cs.CL cs.CR cs.LG math.ST stat.TH 90%

A Statistical Hypothesis Testing Framework for Data Misappropriation Detection in Large Language Models

Yinpeng Cai, Lexin Li, Linjun Zhang

机构 * Peking University（北京大学）； University of California at Berkeley（加州大学伯克利分校）； Rutgers University（罗格斯大学）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 29 pages, 5 figures

2408.13442 2025-09-03 cs.LG cs.AI cs.CL stat.ML 90%

A Law of Next-Token Prediction in Large Language Models

Hangfeng He, Weijie J. Su

机构 * University of Rochester（罗切斯特大学）； University of Pennsylvania（宾夕法尼亚大学）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Transferred for publication to Physical Review E from Physical Review Research (to waive publication charges)

2508.01908 2025-08-05 cs.LG cs.AI cs.CL 90%

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models

Istabrak Abbes, Gopeshh Subbaraj, Matthew Riemer, Nizar Islah, Benjamin Therien, Tsuguchika Tabaru, Hiroaki Kingetsu, Sarath Chandar, Irina Rish

机构 * Université de Montréal（蒙特利尔大学）； Mila – Quebec AI Institute（魁北克人工智能研究院）； Chandar Research Lab（Chandar研究实验室）； IBM Research（IBM研究院）； Fujitsu Research（富士通研究院）； Polytechnique Montréal（蒙特利尔理工学院）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2507.22187 2025-07-31 cs.CL cs.AI cs.LG 90%

A Scalable Pipeline for Estimating Verb Frame Frequencies Using Large Language Models

Adam M. Morgan, Adeen Flinker

机构 * NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）； NYU Tandon School of Engineering（纽约大学坦顿工程学院）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2507.20520 2025-07-29 cs.CL cs.AI cs.CE cs.LG cs.RO 90%

AQUA: A Large Language Model for Aquaculture & Fisheries

Praneeth Narisetty, Uday Kumar Reddy Kattamanchi, Lohit Akshant Nimma, Sri Ram Kaushik Karnati, Shiva Nagendra Babu Kore, Mounika Golamari, Tejashree Nageshreddy

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2506.17311 2025-06-24 cs.CY 90%

Can Large Language Models Be Trusted Paper Reviewers? A Feasibility Study

Chuanlei Li, Xu Hu, Minghui Xu, Kun Li, Yue Zhang, Xiuzhen Cheng

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

2505.07796 2025-06-23 cs.CL cs.AI cs.LG 90%

Learning Dynamics in Continual Pre-Training for Large Language Models

Xingjin Wang, Howe Tissue, Lu Wang, Linjing Li, Daniel Dajun Zeng

机构 * School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China（人工智能学院，中国科学院大学，北京）； State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院，北京）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ICML2025 (Oral)

2505.22232 2025-06-03 cs.CL cs.AI cs.LG 90%

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models

Mehdi Ali, Manuel Brack, Max Lübbering, Elias Wendt, Abbas Goher Khan, Richard Rutmann, Alex Jude, Maurice Kraus, Alexander Arno Weber, David Kaczér, Florian Mai, Lucie Flek, Rafet Sifa, Nicolas Flores-Herr, Joachim Köhler, Patrick Schramowski, Michael Fromm, Kristian Kersting

机构 * Lamarr Institute（拉马尔研究所）； Fraunhofer IAIS（弗劳恩霍夫iais研究所）； DFKI SAINT（DFKI SAINT研究所）； Hessian AI（黑森人工智能研究所）； TU Darmstadt（图宾根技术大学）

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Project page available at https://huggingface.co/spaces/Jackal-AI/JQL

2404.11045 2025-05-29 cs.CL cs.AI cs.LG 90%

Offset Unlearning for Large Language Models

James Y. Huang, Wenxuan Zhou, Fei Wang, Fred Morstatter, Sheng Zhang, Hoifung Poon, Muhao Chen

机构 * University of Southern California（南加州大学）； Microsoft Research（微软研究院）； University of California, Davis（加州大学戴维斯分校）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in TMLR. https://openreview.net/pdf?id=A4RLpHPXCu

2505.15784 2025-05-22 cs.LG cs.AI cs.CL 90%

Large Language Models as Computable Approximations to Solomonoff Induction

Jun Wan, Lingrui Mei

机构 * UBS AG（UBS集团）； State Key Lab of AI Safety（人工智能安全国家重点实验室）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Both authors contributed equally

2501.11496 2025-05-20 cs.CL cs.AI cs.LG 90%

Generative AI and Large Language Models in Language Preservation: Opportunities and Challenges

Vincent Koc

机构 * Hyperthink Labs（Hyperthink实验室）； University of Queensland（昆士兰大学）； University of New South Wales（新南威尔士大学）； Comet ML Inc.（Comet ML公司）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages, 3 figures, 2 tables, submitted for IEEE publication. Pre-print updated as part of review process

2505.04016 2025-05-08 cs.CL cs.AI cs.LG 90%

SLOT: Structuring the Output of Large Language Models

Darren Yow-Bang Wang, Zhengyuan Shen, Soumya Smruti Mishra, Zhichao Xu, Yifei Teng, Haibo Ding

机构 * Amazon Web Services（亚马逊网络服务）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.02639 2025-05-06 cs.LG cs.AI cs.CL 90%

Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning

Xuan Lin, Qingrui Liu, Hongxin Xiang, Daojian Zeng, Xiangxiang Zeng

机构 * School of Computer Science, Xiangtan University（湘潭大学计算机科学学院）； College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）； Institute of AI and Targeted International Communication, Hunan Normal University（湖南师范大学人工智能与定向国际传播研究院）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted for publication at IJCAI 2025

2311.06233 2025-04-29 cs.CL cs.AI cs.LG 90%

Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models

Shahriar Golchin, Mihai Surdeanu

机构 * Department of Computer Science, University of Arizona（计算机科学系，亚利桑那大学）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Final version; accepted at TACL

2504.10000 2025-04-15 cs.CR cs.AI cs.CL cs.CV cs.LG 90%

Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?

Yanbo Wang, Jiyang Guan, Jian Liang, Ran He

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to CVPR 2025, codes in process

2410.13360 2025-03-31 cs.CV cs.AI cs.CL cs.LG cs.MM 90%

RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models

Haoran Hao, Jiaming Han, Changsheng Li, Yu-Feng Li, Xiangyu Yue

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by CVPR 2025. Code: https://github.com/Hoar012/RAP-MLLM

2411.13485 2025-03-11 cs.CL cs.AI cs.LG 90%

Utilizing Large Language Models to Synthesize Product Desirability Datasets

John D. Hastings, Sherri Weitl-Harms, Joseph Doty, Zachary J. Myers, Warren Thompson

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages, 2 figures, 6 tables, updated author list

Journal ref 2024 IEEE International Conference on Big Data (IEEE BigData 2024)

2402.07818 2025-03-11 cs.LG cs.AI cs.CL 90%

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning

Z Liu, J Lou, W Bao, Y Hu, B Li, Z Qin, K Ren

专题命中预训练与数据：language model(title,abstract)；large language model(title)；LLM(abstract)；pretraining(abstract)

2404.08679 2025-03-07 cs.CL cs.AI cs.LG stat.ML 90%

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector

Andi Zhang, Tim Z. Xiao, Weiyang Liu, Robert Bamler, Damon Wischik

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2404.13885 2025-01-17 cs.CY cs.AI cs.CL cs.LG 90%

Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals

Qingyang Wu, Ying Xu, Tingsong Xiao, Yunze Xiao, Yitong Li, Tianyang Wang, Yichi Zhang, Shanghai Zhong, Yuwei Zhang, Wei Lu, Yifan Yang

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2412.15285 2024-12-23 cs.CL cs.AI cs.LG 90%

Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining

Steven Feng, Shrimai Prabhumoye, Kezhi Kong, Dan Su, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro

专题命中预训练与数据：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

2412.01014 2024-12-03 cs.LG cs.AI cs.CL 90%

Detecting Memorization in Large Language Models

Eduardo Slonski

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2411.07990 2024-11-13 cs.CL cs.AI cs.LG 90%

Derivational Morphology Reveals Analogical Generalization in Large Language Models

Valentin Hofmann, Leonie Weissweiler, David Mortensen, Hinrich Schütze, Janet Pierrehumbert

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2410.05269 2024-10-08 cs.CL cs.AI cs.LG 90%

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models

Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2024 Main Conference. Project website: https://feiwang96.github.io/DataAdvisor/

2403.08763 2024-09-05 cs.LG cs.AI cs.CL 90%

Simple and Scalable Strategies to Continually Pre-train Large Language Models

Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L. Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, Irina Rish

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

1. 预训练与数据 12393 篇

Revisiting Multilingual Data Mixtures in Language Model Pretraining

LayerIF: Estimating Layer Quality for Large Language Models using Influence Functions

GUARD: Guided Unlearning and Retention via Data Attribution for Large Language Models

Synergistic Enhancement of Requirement-to-Code Traceability: A Framework Combining Large Language Model based Data Augmentation and an Advanced Encoder

Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation

A Statistical Hypothesis Testing Framework for Data Misappropriation Detection in Large Language Models

A Law of Next-Token Prediction in Large Language Models

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models

A Scalable Pipeline for Estimating Verb Frame Frequencies Using Large Language Models

AQUA: A Large Language Model for Aquaculture & Fisheries

Can Large Language Models Be Trusted Paper Reviewers? A Feasibility Study

Learning Dynamics in Continual Pre-Training for Large Language Models

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models

Offset Unlearning for Large Language Models

Large Language Models as Computable Approximations to Solomonoff Induction

Generative AI and Large Language Models in Language Preservation: Opportunities and Challenges

SLOT: Structuring the Output of Large Language Models

Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning

Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models

Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?

RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models

Utilizing Large Language Models to Synthesize Product Desirability Datasets

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector

Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals

Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining

Detecting Memorization in Large Language Models

Derivational Morphology Reveals Analogical Generalization in Large Language Models

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models

Simple and Scalable Strategies to Continually Pre-train Large Language Models