语言大模型 / LLM

2510.24139 2025-10-29 cs.CL cs.AI 84%

Beyond Line-Level Filtering for the Pretraining Corpora of LLMs

Chanwoo Park, Suyoung Park, Yelim Ahn, Jongmin Kim, Jongyeon Park, Jaejin Lee

专题命中预训练与数据：pretraining(title)；language model(abstract)；small language model(abstract)；分类 cs.CL、cs.AI

Comments submitted to ACL ARR Rolling Review

2510.24159 2025-10-29 astro-ph.GA astro-ph.IM cs.AI cs.LG 81%

Self-supervised Synthetic Pretraining for Inference of Stellar Mass Embedded in Dense Gas

Keiya Hirashima, Shingo Nozaki, Naoto Harada

机构 * Center for Interdisciplinary Theoretical and Mathematical Sciences (iTHEMS) RIKEN（交叉学科理论与数学科学中心（iTHEMS）RIKEN）； Department of Earth and Planetary Sciences（地球与行星科学系）； Department of Astronomy（天文学系）

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments 6 pages, 3 figures, 1 table, accepted for NeurIPS 2025 ML4PS workshop

2510.15870 2025-10-29 cs.CV cs.AI cs.CL 81%

OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM

Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, Ligeng Zhu, Yuanhang Su, Sean Lin, An-Chieh Cheng, Zhen Wan, Jinchuan Tian, Yuming Lou, Dong Yang, Zhijian Liu, Yukang Chen, Ambrish Dantrey, Ehsan Jahangiri, Sreyan Ghosh, Daguang Xu, Ehsan Hosseini-Asl, Danial Mohseni Taheri, Vidya Murali, Sifei Liu, Yao Lu, Oluwatobi Olabiyi, Yu-Chiang Frank Wang, Rafael Valle, Bryan Catanzaro, Andrew Tao, Song Han, Jan Kautz, Hongxu Yin, Pavlo Molchanov

机构 * NVIDIA

专题命中预训练与数据：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments Technical Report. Code: https://github.com/NVlabs/OmniVinci

2404.13397 2025-10-29 cs.CL cs.AI 73%

Retrieval-Augmented Generation-based Relation Extraction

Sefika Efeoglu, Adrian Paschke

机构 * Electrical Engineering and Computer Science Department, Technische Universitaet Berlin（技术大学柏林电子工程与计算机科学系）； Department of Computer Science, Freie Universitaet Berlin（柏林自由大学计算机科学系）； Data Analytic Center (DANA), Fraunhofer Institute FOKUS（弗劳恩霍夫研究所FOKUS数据分析中心）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments published at the Semantic Web journal. The last version is available: https://doi.org/10.1177/22104968251385519

2510.24019 2025-10-29 cs.SE cs.AI 70%

Lifecycle-Aware code generation: Leveraging Software Engineering Phases in LLMs

Xing Xing, Wei Wang, Lipeng Ma, Weidong Yang, Junjie Zheng

机构 * School of Computer Science, Fudan University（复旦大学计算机科学学院）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.AI

2506.01855 2025-10-29 cs.LG cs.IT math.IT stat.ML 70%

Trade-offs in Data Memorization via Strong Data Processing Inequalities

Vitaly Feldman, Guy Kornowski, Xin Lyu

机构 * Apple（苹果公司）； Weizmann Institute of Science（魏茨曼科学研究院）； UC Berkeley（伯克利大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Appeared in COLT 2025; this revision includes an improved upper bound in Theorem 3.1, as well as several minor clarifications and modifications

2510.24365 2025-10-29 cs.CL 57%

Text Simplification with Sentence Embeddings

Matthew Shardlow

机构 * Manchester Metropolitan University（曼彻斯特 Metropolitan 大学）

专题命中预训练与数据：LLM(abstract)；分类 cs.CL

2405.07883 2025-10-29 cs.CL 57%

Zero-Shot Tokenizer Transfer

Benjamin Minixhofer, Edoardo Maria Ponti, Ivan Vulić

机构 * University of Cambridge（剑桥大学）； University of Edinburgh（爱丁堡大学）

专题命中预训练与数据：language model(abstract)；分类 cs.CL

Comments NeurIPS 2024

2510.24261 2025-10-29 cs.RO cs.AI cs.CV 57%

DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulation

Jingyi Tian, Le Wang, Sanping Zhou, Sen Wang, Jiayi Li, Gang Hua

机构 * National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家重点实验室）； National Engineering Research Center for Visual Information and Applications（视觉信息与应用国家工程研究中心）； Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）； Xi’an Jiaotong University（西安交通大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI

Comments Accepted to NeurIPS 2025

2510.24053 2025-10-29 cs.LG q-bio.QM 57%

Low-N Protein Activity Optimization with FolDE

Jacob B. Roberts, Catherine R. Ji, Isaac Donnell, Thomas D. Young, Allison N. Pearson, Graham A. Hudson, Leah S. Keiser, Mia Wesselkamper, Peter H. Winegar, Janik Ludwig, Sarah H. Klass, Isha V. Sheth, Ezechinyere C. Ukabiala, Maria C. T. Astolfi, Benjamin Eysenbach, Jay D. Keasling

专题命中预训练与数据：language model(abstract)；分类 cs.LG

Comments 18 pages, 4 figures. Preprint. Open-source software available at https://github.com/JBEI/foldy

2510.24703 2025-10-29 physics.med-ph 50%

Cluster Dose Prediction in Carbon Ion Therapy: Using Transfer Learning from a Pretrained Dose Prediction U-Net

Miriam Schwarze, Hui Khee Looe, Björn Poppe, Leo Thomas, Hans Rabus

专题命中预训练与数据：pretraining(abstract)

2405.07046 2025-10-29 cs.CV 50%

RETTA: Retrieval-Enhanced Test-Time Adaptation for Zero-Shot Video Captioning

Yunchuan Ma, Laiyun Qing, Guorong Li, Yuankai Qi, Amin Beheshti, Quan Z. Sheng, Qingming Huang

机构 * University of Chinese Academy of Science, Beijing,100190, China（中国科学院大学）； Macquarie University（麦考瑞大学）

专题命中预训练与数据：language model(abstract)

Comments Published in Pattern Recognition

2411.09539 2025-10-29 cs.CL cs.AI cs.LG 89%

Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide

Marton Szep, Daniel Rueckert, Rüdiger von Eisenhart-Rothe, Florian Hinterwimmer

机构 * Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital（人工智能在医疗与医学中的chair，技术大学慕尼黑（TUM）和慕尼黑技术大学医院）； Department of Orthopaedics and Sports Orthopaedics, TUM University Hospital（骨科与运动骨科，慕尼黑技术大学医院）； Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML））； Department of Computing, Imperial College London, UK（计算系，帝国理工学院伦敦）

专题命中指令微调：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to TACL. Pre-MIT Press version. Major restructuring; added preference alignment section and additional tables. 36 pages

2505.10978 2025-10-29 cs.LG cs.AI 86%

Group-in-Group Policy Optimization for LLM Agent Training

Lang Feng, Zhenghai Xue, Tingcong Liu, Bo An

机构 * Nanyang Technological University（南洋理工大学）； Skywork AI

专题命中指令微调：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2025

2510.24331 2025-10-29 cs.LG cs.CV 85%

What do vision-language models see in the context? Investigating multimodal in-context learning

Gabriel O. dos Santos, Esther Colombini, Sandra Avila

机构 * Instituto de Computação, Universidade Estadual de Campinas (UNICAMP)（计算机学院，Campinas州立大学（UNICAMP））

专题命中指令微调：language model(title,abstract)；large language model(abstract)；instruction tuning(abstract)；分类 cs.LG

2510.05024 2025-10-29 cs.LG 85%

Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment

Nevan Wichers, Aram Ebtekar, Ariana Azarbal, Victor Gillioz, Christine Ye, Emil Ryd, Neil Rathi, Henry Sleight, Alex Mallen, Fabien Roger, Samuel Marks

专题命中指令微调：prompting(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments v2 Updates references. v3 Updates references; Adds IFEval results; Improves appendix readability; Adds author contributions

2510.24034 2025-10-29 cs.CV 85%

AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts

Yufan Liu, Wanqian Zhang, Huashan Chen, Lin Wang, Xiaojun Jia, Zheng Lin, Weiping Wang

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）； School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）； School of Cyberspace, Hangzhou Dianzi University（杭州电子科技大学网络空间学院）； Nanyang Technological University（南洋理工大学）

专题命中指令微调：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by ICCV 2025

2510.24285 2025-10-29 cs.CV cs.AI cs.CL 84%

ViPER: Empowering the Self-Evolution of Visual Perception Abilities in Vision-Language Model

Juntian Zhang, Song Jin, Chuanqi Cheng, Yuhan Liu, Yankai Lin, Xun Zhang, Yufei Zhang, Fei Jiang, Guojun Yin, Wei Lin, Rui Yan

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学人工智能学院）； Meituan（美团）； MBZUAI ； Wuhan University（武汉大学）

专题命中指令微调：language model(title,abstract)；SFT(abstract)；分类 cs.CL、cs.AI

2508.19563 2025-10-29 cs.LG cs.AI stat.AP stat.ML 82%

Robustness is Important: Limitations of LLMs for Data Fitting

Hejia Liu, Mochen Yang, Gediminas Adomavicius

机构 * Carlson School of Management, University of Minnesota（明尼苏达大学卡尔森管理学院）

专题命中指令微调：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

2506.00799 2025-10-29 cs.LG 81%

Uni-LoRA: One Vector is All You Need

Kaiyang Li, Shaobo Han, Qing Su, Wei Li, Zhipeng Cai, Shihao Ji

机构 * School of Computing University of Connecticut（计算机学院美国康涅狄格大学）； NEC Labs America（NEC美国实验室）； Dept. of Computer Science Georgia State University（计算机科学系美国佐治亚州立大学）

专题命中指令微调：LLM(abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments NeurIPS 2025 Spotlight

2503.23502 2025-10-29 cs.CV cs.AI cs.LG cs.RO 81%

Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model

Jannik Endres, Oliver Hahn, Charles Corbière, Simone Schaub-Meyer, Stefan Roth, Alexandre Alahi

机构 * École Polytechnique Fédérale de Lausanne (EPFL)（瑞士联邦理工学院）； Technical University of Darmstadt, Department of Computer Science（德累斯顿技术大学计算机科学系）

专题命中指令微调：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted at IROS 2025. Project page: https://vita-epfl.github.io/DFI-OmniStereo-website/

2510.24619 2025-10-29 cs.CL cs.AI cs.LG 80%

Zero-Shot Cross-Lingual Transfer using Prefix-Based Adaptation

Snegha A, Sayambhu Sen, Piyush Singh Pasi, Abhishek Singhania, Preethi Jyothi

机构 * Indian Institute of Technology Bombay（印度理工学院班加罗尔分校）

专题命中指令微调：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 Pages

2509.06463 2025-10-29 cs.AI 79%

Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Set

Chengwei Wu, Li Du, Hanyu Zhao, Yiming Ju, Jiapu Wang, Tianyu Chen, Haoyi Zhou

机构 * Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）； Beijing University of Technology（北京理工大学）； Beihang University（北航）

专题命中指令微调：LLM(title)；SFT(abstract)；分类 cs.AI

2411.12118 2025-10-29 cs.LG cs.CL 79%

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers

Tiberiu Musat

机构 * ETH Zürich（苏黎世联邦理工学院）

专题命中指令微调：large language model(abstract)；language model(abstract)；prompting(abstract)；分类 cs.CL、cs.LG

Journal ref International Conference on Learning Representations (ICLR), 2025

2510.24061 2025-10-29 cs.LG cs.AI 79%

FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic

Kanghyun Choi, Hyeyoon Lee, SunJong Park, Dain Kwon, Jinho Lee

机构 * Department of Electrical and Computer Engineering（电气与计算机工程系）； Seoul National University（首尔国立大学）

专题命中指令微调：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2025

2510.24020 2025-10-29 cs.CL cs.AI 79%

Teaching LLMs to Abstain via Fine-Grained Semantic Confidence Reward

Hao An, Yang Xu

机构 * Computational Linguistics and Consciousness Sciences Lab, Southern University of Science and Technology, China（南方科技大学计算语言学与意识科学实验室）

专题命中指令微调：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 23pages, 4figures

2510.23891 2025-10-29 cs.CR cs.AI cs.LG 79%

PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs

Jiaqi Xue, Yifei Zhao, Mansour Al Ghanim, Shangqian Gao, Ruimin Sun, Qian Lou, Mengxin Zheng

机构 * University of Central Florida（佛罗里达中央大学）； Florida State University（佛罗里达州立大学）； Florida International University（佛罗里达国际大学）

专题命中指令微调：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2510.21729 2025-10-29 cs.IR cs.AI 77%

CustomIR: Unsupervised Fine-Tuning of Dense Embeddings for Known Document Corpora

Nathan Paull

机构 * Valkyrie Andromeda（瓦尔基里安德墨达）

专题命中指令微调：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2510.24161 2025-10-29 cs.AI cs.MM cs.RO 77%

BLM$_1$: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning

Wentao Tan, Bowen Wang, Heng Zhi, Chenyu Liu, Zhe Li, Jian Liu, Zengrong Lin, Yukun Dai, Yipeng Chen, Wenjie Yang, Enci Xie, Hao Xue, Baixu Ji, Chen Xu, Zhibin Wang, Tianshi Wang, Lei Zhu, Heng Tao Shen

机构 * School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）

专题命中指令微调：large language model(abstract)；language model(abstract)；foundation model(abstract)；分类 cs.AI

2510.23818 2025-10-29 cs.LG 70%

ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning

Yilang Zhang, Xiaodong Yang, Yiwei Cai, Georgios B. Giannakis

机构 * ML Research, Morgan Stanley, New York, NY 10019, USA（ML研究，摩根大通，纽约，纽约州，10019，美国）； Visa Research, Austin, TX 78759, USA（Visa研究，奥斯汀，德克萨斯州，78759，美国）； Department of ECE, University of Minnesota, MN 55455, USA（电子工程系，明尼苏达大学，明尼苏达州，55455，美国）

专题命中指令微调：large language model(abstract)；language model(abstract)；分类 cs.LG

1. 预训练与数据 12 篇