语言大模型 / LLM - arXivDaily 专题

2510.14702 2025-10-17 cs.AI 90%

Cognitive-Aligned Spatio-Temporal Large Language Models For Next Point-of-Interest Prediction

Penglong Zhai, Jie Li, Fanyi Di, Yue Liu, Yifang Yuan, Jie Huang, Peng Wu, Sicong Wang, Mingyang Yin, Tingting Hu, Yao Xu, Xin Li

机构 * AMAP, Alibaba Group（阿里集团AMAP）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；SFT(abstract)

Comments 12 pages, 5 figures

2510.13915 2025-10-17 cs.CL cs.AI cs.LG 89%

Readability $\ne$ Learnability: Rethinking the Role of Simplicity in Training Small Language Models

Ivan Lee, Taylor Berg-Kirkpatrick

机构 * UC San Diego（圣地亚哥大学）

专题命中预训练与数据：language model(title,abstract)；small language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to COLM 2025 (Spotlight)

2507.18671 2025-10-17 cs.LG cs.AI 88%

Innovator: Scientific Continued Pretraining with Fine-grained MoE Upcycling

Ning Liao, Xiaoxing Wang, Zehao Lin, Weiyang Guo, Feng Hong, Shixiang Song, Geng Yu, Zihua Zhao, Sitao Xie, Longxuan Wei, Xiangqi Jin, Xiaohan Qin, Jiale Ma, Kai Chen, Jiangchao Yao, Zhouhan Lin, Junchi Yan, Zhiyu Li, Feiyu Xiong, Yanfeng Wang, Linfeng Zhang

机构 * Shanghai Jiao Tong University（上海交通大学）； MemTensor (Shanghai) Technology Co., Ltd.（MemTensor（上海）技术有限公司）； Institute for Advanced Algorithms Research, Shanghai.（上海先进算法研究所）

专题命中预训练与数据：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Technical Report

2501.18962 2025-10-17 cs.LG 87%

Spend Wisely: Maximizing Post-Training Gains in Iterative Synthetic Data Bootstrapping

Pu Yang, Yunzhen Feng, Ziyuan Chen, Yuhang Wu, Zhuoyuan Li

机构 * Peking University（北京大学）； New York University（纽约大学）； UC Berkeley（加州大学伯克利分校）； National University of Singapore（新加坡国立大学）

专题命中预训练与数据：post-training(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments NeurIPS 2025 (spotlight)

2510.13996 2025-10-17 cs.CL 85%

The German Commons - 154 Billion Tokens of Openly Licensed Text for German Language Models

Lukas Gienapp, Christopher Schröder, Stefan Schweter, Christopher Akiki, Ferdinand Schlatt, Arden Zimmermann, Phillipe Genêt, Martin Potthast

机构 * University of Kassel, hessian.AI（卡塞尔大学、hessian.AI）； Independent Researcher（独立研究者）； Leipzig University（莱比锡大学）

专题命中预训练与数据：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL

Comments 13 pages, 3 figures, 12 tables, includes datasheet

2403.15356 2025-10-17 cs.CV 82%

Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation

Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J. Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, Xiao Xiang Zhu

机构 * Chair of Data Science in Earth Observation, Technical University of Munich (TUM)（地球观测数据科学教授职位，慕尼黑技术大学）； Munich Center for Machine Learning（慕尼黑机器学习中心）； AIML Lab, School of Computer Science, University of St. Gallen（人工智能实验室，圣加尔登大学计算机科学学院）； School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens（农村、测绘与地理信息工程学院，国家技术大学雅典）； Image Processing Laboratory (IPL), Universitat de València（图像处理实验室（IPL），瓦伦西亚大学）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(abstract)

Comments 18 pages, 8 figures

2507.21750 2025-10-17 cs.CL 79%

Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removal

Yang Wang, Chenghao Xiao, Yizhi Li, Stuart E. Middleton, Noura Al Moubayed, Chenghua Lin

机构 * The University of Manchester, UK（曼彻斯特大学）； Durham University, UK（杜伦大学）； The University of Southampton, UK（南安普顿大学）； Automated Analytics, UK（自动化分析）

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments This paper was accepted with an A-decision to Transactions of the Association for Computational Linguistics. This version is the pre-publication version prior to MIT Press production

2510.13859 2025-10-17 cs.SE cs.AI 77%

Benchmarking Correctness and Security in Multi-Turn Code Generation

Ruchit Rawal, Jeffrey Yang Fan Chiang, Chihao Shen, Jeffery Siyuan Tian, Aastha Mahajan, Tom Goldstein, Yizheng Chen

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2510.14919 2025-10-17 cs.CL cs.AI cs.LG 75%

Predicting Task Performance with Context-aware Scaling Laws

Kyle Montgomery, David Park, Jianhong Tu, Michael Bendersky, Beliz Gunel, Dawn Song, Chenguang Wang

机构 * UC Santa Cruz（加州大学圣克ruz分校）； Washington University in St. Louis（华盛顿大学圣路易斯分校）； Databricks（Databricks公司）； Google DeepMind（谷歌DeepMind）； UC Berkeley（加州大学伯克利分校）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2412.04445 2025-10-17 cs.RO cs.AI cs.CL cs.CV cs.LG 75%

Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos

Yi Chen, Yuying Ge, Weiliang Tang, Yizhuo Li, Yixiao Ge, Mingyu Ding, Ying Shan, Xihui Liu

机构 * The University of Hong Kong（香港大学）； ARC Lab, Tencent PCG（腾讯PCG实验室）； The Chinese University of Hong Kong（香港中文大学）； University of California, Berkeley（加州大学伯克利分校）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICCV 2025. Project page: https://chenyi99.github.io/moto/

2510.14915 2025-10-17 cs.CL 70%

Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation

Xujun Peng, Anoop Kumar, Jingyu Wu, Parker Glenn, Daben Liu

机构 * AI Foundations, Capital One McLean, VA, USA（AI基础研究，Capital One麦莱恩，美国弗吉尼亚州）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments EMNLP 2025 Industry track

2510.13992 2025-10-17 cs.SE cs.LG 70%

Signature in Code Backdoor Detection, how far are we?

Quoc Hung Le, Thanh Le-Cong, Bach Le, Bowen Xu

机构 * North Carolina State University（北卡罗来纳州立大学）； The University of Melbourne（墨尔本大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 20 pages, 3 figures

2510.14968 2025-10-17 cs.RO cs.AI cs.CV cs.LG cs.SY eess.SY 62%

RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks

Mingxuan Yan, Yuping Wang, Zechun Liu, Jiachen Li

机构 * University of California, Riverside（加州大学河滨分校）； University of Michigan（密歇根大学）； Meta AI

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

Comments 39th Conference on Neural Information Processing Systems (NeurIPS 2025); Project Website: rdd-neurips.github.io

2510.14139 2025-10-17 cs.LG cs.AI 62%

Inferred global dense residue transition graphs from primary structure sequences enable protein interaction prediction via directed graph convolutional neural networks

Islam Akef Ebeid, Haoteng Tang, Pengfei Gu

机构 * Division of Computer Science Texas Woman’s University（计算机科学系德克萨斯女子大学）； Department of Computer Science The University of Texas Rio Grande Valley（计算机科学系德克萨斯大学里奥格兰德谷分校）

专题命中预训练与数据：language model(abstract)；分类 cs.AI、cs.LG

Comments under review in Frontiers in Bioinformatics

2510.13913 2025-10-17 cs.CL cs.AI 62%

Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanisms

Shrey Pandit, Xuan-Phi Nguyen, Yifei Ming, Austin Xu, Jiayu Wang, Caiming Xiong, Shafiq Joty

机构 * Salesforce AI Research（Salesforce AI研究院）； University of Wisconsin-Madison（威斯康星大学麦迪逊分校）

专题命中预训练与数据：language model(abstract)；分类 cs.CL、cs.AI

Comments Preprint. ICLR 26 submission

2510.14648 2025-10-17 cs.CV cs.AI 57%

In-Context Learning with Unpaired Clips for Instruction-based Video Editing

Xinyao Liao, Xianfang Zeng, Ziye Song, Zhoujie Fu, Gang Yu, Guosheng Lin

机构 * Nanyang Technological University（南洋理工大学）； StepFun

专题命中预训练与数据：pretraining(abstract)；分类 cs.AI

2510.14411 2025-10-17 cs.LG cs.MM cs.SD eess.AS 57%

Revisit Modality Imbalance at the Decision Layer

Xiaoyu Ma, Hao Chen

机构 * School of Computer Science and Engineering, Southeast University, Nanjing, China（计算机科学与工程学院，东南大学，南京，中国）； Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（新一代人工智能技术及其交叉应用重点实验室（东南大学），教育部，中国）

专题命中预训练与数据：pretraining(abstract)；分类 cs.LG

Comments Some Insights in Balanced Multimodal Learning

2505.24834 2025-10-17 cs.CL 57%

Multilinguality Does not Make Sense: Investigating Factors Behind Zero-Shot Transfer in Sense-Aware Tasks

Roksana Goworek, Haim Dubossarsky

机构 * Queen Mary University of London（伦敦玛丽女王大学）； The Alan Turing Institute（艾伦·图灵研究所）； University of Cambridge（剑桥大学）

专题命中预训练与数据：pretraining(abstract)；分类 cs.CL

Comments accepted to EMNLP 2025 Main

2509.18092 2025-10-17 cs.CV 50%

ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation

Guocheng Gordon Qian, Daniil Ostashev, Egor Nemchinov, Avihay Assouline, Sergey Tulyakov, Kuan-Chieh Jackson Wang, Kfir Aberman

机构 * Snap Inc.

专题命中预训练与数据：prompting(abstract)

Comments Accepted to SIGGRAPH Asia 2025, webpage: https://snap-research.github.io/composeme/

1. 预训练与数据 19 篇

Cognitive-Aligned Spatio-Temporal Large Language Models For Next Point-of-Interest Prediction

Readability $\ne$ Learnability: Rethinking the Role of Simplicity in Training Small Language Models

Innovator: Scientific Continued Pretraining with Fine-grained MoE Upcycling

Spend Wisely: Maximizing Post-Training Gains in Iterative Synthetic Data Bootstrapping

The German Commons - 154 Billion Tokens of Openly Licensed Text for German Language Models

Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation

Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removal

Benchmarking Correctness and Security in Multi-Turn Code Generation

Predicting Task Performance with Context-aware Scaling Laws

Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos

Harmonizing Diverse Models: A Layer-wise Merging Strategy for Consistent Generation

Signature in Code Backdoor Detection, how far are we?

RDD: Retrieval-Based Demonstration Decomposer for Planner Alignment in Long-Horizon Tasks

Inferred global dense residue transition graphs from primary structure sequences enable protein interaction prediction via directed graph convolutional neural networks

Synthesizing Agentic Data for Web Agents with Progressive Difficulty Enhancement Mechanisms

In-Context Learning with Unpaired Clips for Instruction-based Video Editing

Revisit Modality Imbalance at the Decision Layer

Multilinguality Does not Make Sense: Investigating Factors Behind Zero-Shot Transfer in Sense-Aware Tasks

ComposeMe: Attribute-Specific Image Prompts for Controllable Human Image Generation