语言大模型 / LLM - arXivDaily 专题

2510.27254 2025-11-03 cs.CL cs.AI cs.LG 80%

Languages are Modalities: Cross-Lingual Alignment via Encoder Injection

Rajan Agarwal, Aarush Gupta

机构 * University of Waterloo（多伦多大学）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 14 pages, 3 Figures

2510.26345 2025-10-31 cs.CL cs.AI cs.LG 80%

MisSynth: Improving MISSCI Logical Fallacies Classification with Synthetic Data

Mykhailo Poliakov, Nadiya Shvai

机构 * National University of Kyiv-Mohyla Academy（基辅-莫希拉学院国家大学）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.19866 2025-09-30 cs.AI cs.CL cs.LG 80%

HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation

Feng Xiong, Hongling Xu, Yifei Wang, Runxi Cheng, Yong Wang, Xiangxiang Chu

机构 * Alibaba Group（阿里巴巴集团）； University of Chinese Academy of Sciences（中国科学院大学）； Tsinghua University（清华大学）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2509.19249 2025-09-26 cs.CL cs.AI cs.LG 80%

Reinforcement Learning on Pre-Training Data

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Evander Yang, Kun Li, Haoyuan Wu, Jiajia Wu, Zihao Zheng, Chenchen Zhang, Kun Shi, Kyrierl Deng, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang Xu, Zheng Fang, Shuai Li, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bo Chao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Wayyt Wang, Bo Zhou, Di Wang

机构 * LLM Department, Tencent（腾讯大模型部门）； HunYuan Infra Team（文心一言基础设施团队）； The Chinese University of Hong Kong（香港中文大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Work in progress

2305.12766 2025-09-16 cs.CL cs.AI cs.LG 80%

Understanding Emergent In-Context Learning from a Kernel Regression Perspective

Chi Han, Ziqi Wang, Han Zhao, Heng Ji

机构 * Siebel School of Computing and Data Science（计算与数据科学学院）； University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Transactions on Machine Learning Research (TMLR 2025)

2408.11813 2025-09-08 cs.CV 80%

SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs

Yuanyang Yin, Yaqi Zhao, Yajie Zhang, Yuanxing Zhang, Ke Lin, Jiahao Wang, Xin Tao, Pengfei Wan, Wentao Zhang, Feng Zhao

机构 * University of Science and Technology of China（中国科学技术大学）； Peking University（北京大学）； Kuaishou Technology（快手科技）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

2508.21824 2025-09-01 cs.CV 80%

DriveQA: Passing the Driving Knowledge Test

Maolin Wei, Wanzhou Liu, Eshed Ohn-Bar

机构 * Boston University（波士顿大学）； Washington University in St. Louis（华盛顿大学圣路易斯分校）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted by ICCV 2025. Project page: https://driveqaiccv.github.io/

2507.15788 2025-07-22 cs.LG cs.AI cs.CL 80%

Small LLMs Do Not Learn a Generalizable Theory of Mind via Reinforcement Learning

Sneheel Sarangi, Hanan Salam

机构 * NYU Abu Dhabi（纽约大学阿布扎比分校）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

2503.02951 2025-07-15 cs.LG cs.AI cs.CL 80%

KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding

Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, Radha Poovendran

机构 * Microsoft GenAI（微软生成人工智能）； University of Washington（华盛顿大学）； The University of Texas at Austin（德克萨斯大学奥斯汀分校）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by ACL 2025. Codes and Data: https://kodcode-ai.github.io/

2507.08284 2025-07-14 cs.LG cs.AI cs.CL 80%

Lightweight Safety Guardrails via Synthetic Data and RL-guided Adversarial Training

Aleksei Ilin, Gor Matevosyan, Xueying Ma, Vladimir Eremin, Suhaa Dada, Muqun Li, Riyaaz Shaik, Haluk Noyan Tokgozoglu

机构 * Apple（苹果公司）

专题命中预训练与数据：LLM(abstract)；language model(abstract)；small language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2507.00754 2025-07-10 cs.CV 80%

Language-Unlocked ViT (LUViT): Empowering Self-Supervised Vision Transformers with LLMs

Selim Kuzucu, Muhammad Ferjad Naeem, Anna Kukleva, Federico Tombari, Bernt Schiele

机构 * Max Planck Institute for Informatics（马克斯·普朗克信息研究所）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 26 pages, 6 figures

2506.15025 2025-06-19 cs.LG cs.AI cs.CL stat.ML 80%

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size

Soufiane Hayou, Liyuan Liu

机构 * Simons Institute UC Berkeley（Simons研究所加州大学伯克利分校）； Microsoft Research（微软研究院）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments TD,LR: How to set the learning rate for emebdding layer in LLMs?

2407.15621 2025-06-19 cs.CL cs.AI cs.LG 80%

RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering

Soroosh Tayebi Arasteh, Mahshad Lotfinia, Keno Bressem, Robert Siepmann, Lisa Adams, Dyke Ferber, Christiane Kuhl, Jakob Nikolas Kather, Sven Nebelung, Daniel Truhn

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in Radiology: Artificial Intelligence

Journal ref Radiology: Artificial Intelligence, (2025), 7(4):e240476

2505.07859 2025-06-12 cs.CL cs.AI cs.LG 80%

Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective

Daniel Franzen, Jan Disselhoff, David Hartmann

机构 * Johannes Gutenberg University Mainz（莱茵河畔明斯特大学）； Lambda, Inc.（Lambda公司）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2025 camera-ready; 15 pages, 6 figures, 5 tables

2410.22944 2025-06-06 cs.LG cs.AI cs.CL 80%

Focus On This, Not That! Steering LLMs with Adaptive Feature Specification

Tom A. Lamb, Adam Davies, Alasdair Paren, Philip H. S. Torr, Francesco Pinto

专题命中预训练与数据：large language model(abstract)；language model(abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 36pages, 19 figures

2506.02298 2025-06-04 cs.CL cs.AI cs.LG 80%

LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback

Thai Hoang, Kung-Hsiang Huang, Shirley Kokane, Jianguo Zhang, Zuxin Liu, Ming Zhu, Jake Grigsby, Tian Lan, Michael S Ryoo, Chien-Sheng Wu, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong, Juan Carlos Niebles

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments LAM Simulator framework for agentic data generation

2412.12276 2025-06-03 cs.CL cs.AI cs.LG 80%

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective

Seungwook Han, Jinyeop Song, Jeff Gore, Pulkit Agrawal

专题命中预训练与数据：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments https://charming-centaur-089.notion.site/Emergence-and-Effectiveness-of-Task-Vectors-in-In-Context-Learning-An-Encoder-Decoder-Perspective-2054664a1d59814f8401cded3332fce4

2504.03553 2025-05-30 cs.CL cs.AI cs.CV cs.LG cs.MA 80%

Agentic Knowledgeable Self-awareness

Shuofei Qiao, Zhisong Qiu, Baochang Ren, Xiaobin Wang, Xiangyuan Ru, Ningyu Zhang, Xiang Chen, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2025

2501.11478 2025-05-27 cs.CL cs.AI cs.LG 80%

Each Graph is a New Language: Graph Learning with LLMs

Huachi Zhou, Jiahe Du, Chuang Zhou, Chang Yang, Yilin Xiao, Yuxuan Xie, Xiao Huang

机构 * First Author Affiliation（第一作者机构）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.18121 2025-05-26 cs.AI cs.CL cs.LG 80%

ProgRM: Build Better GUI Agents with Progress Rewards

Danyang Zhang, Situo Zhang, Ziyue Yang, Zichen Zhu, Zihan Zhao, Ruisheng Cao, Lu Chen, Kai Yu

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.17140 2025-05-26 cs.CL cs.AI cs.LG 80%

Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs

Essa Jan, Moiz Ali, Muhammad Saram Hassan, Fareed Zaffar, Yasir Zaki

机构 * Lahore University of Management Sciences（拉瓦尔大学管理科学学院）； New York University Abu Dhabi（纽约大学阿布扎克分校）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 4 pages, 1 figure

2505.10066 2025-05-16 cs.CL cs.AI cs.CR cs.LG 80%

Dark LLMs: The Growing Threat of Unaligned AI Models

Michael Fire, Yitzhak Elbazis, Adi Wasenstein, Lior Rokach

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.08194 2025-05-14 cs.RO 80%

CLTP: Contrastive Language-Tactile Pre-training for 3D Contact Geometry Understanding

Wenxuan Ma, Xiaoge Cao, Yixiang Zhang, Chaofan Zhang, Shaobo Yang, Peng Hao, Bin Fang, Yinghao Cai, Shaowei Cui, Shuo Wang

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）； Beihang University（北航）； Beijing University of Posts and Telecommunications（北京邮电大学）； Samsung Research China（三星中国研究院）

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 16 pages

2504.18425 2025-04-28 eess.AS cs.AI cs.CL cs.LG cs.MM cs.SD 80%

Kimi-Audio Technical Report

KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, Jun Chen, Yanru Chen, Yulun Du, Weiran He, Zhenxing Hu, Guokun Lai, Qingcheng Li, Yangyang Liu, Weidong Sun, Jianzhou Wang, Yuzhi Wang, Yuefeng Wu, Yuxin Wu, Dongchao Yang, Hao Yang, Ying Yang, Zhilin Yang, Aoxiong Yin, Ruibin Yuan, Yutong Zhang, Zaida Zhou

机构 * Kimi Team（Kimi团队）

专题命中预训练与数据：LLM(abstract)；foundation model(abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

2504.03160 2025-04-18 cs.AI cs.CL cs.LG 80%

DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments

Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, Pengfei Liu

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2410.08527 2025-04-09 cs.CL cs.AI cs.LG 80%

Scaling Laws for Predicting Downstream Performance in LLMs

Yangyi Chen, Binxuan Huang, Yifan Gao, Zhengyang Wang, Jingfeng Yang, Heng Ji

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to TMLR

2408.11852 2025-03-20 cs.CL cs.AI cs.LG 80%

Fast Training Dataset Attribution via In-Context Learning

Milad Fotouhi, Mohammad Taha Bahadori, Oluwaseyi Feyisetan, Payman Arabshahi, David Heckerman

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2503.12811 2025-03-18 cs.LG cs.AI cs.CL stat.ML 80%

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

Kairong Luo, Haodong Wen, Shengding Hu, Zhenbo Sun, Zhiyuan Liu, Maosong Sun, Kaifeng Lyu, Wenguang Chen

专题命中预训练与数据：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

2502.07164 2025-03-18 cs.CL cs.AI cs.LG 80%

Does Training on Synthetic Data Make Models Less Robust?

Lingze Zhang, Ellie Pavlick

专题命中预训练与数据：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2404.08417 2025-02-11 cs.LG cs.AI cs.CL 80%

AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees

William Fleshman, Aleem Khan, Marc Marone, Benjamin Van Durme

专题命中预训练与数据：large language model(abstract)；language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments In Proceedings of the Conference on Applied Machine Learning in Information Security, 2024

1. 预训练与数据 12429 篇

Languages are Modalities: Cross-Lingual Alignment via Encoder Injection

MisSynth: Improving MISSCI Logical Fallacies Classification with Synthetic Data

HS-STaR: Hierarchical Sampling for Self-Taught Reasoners via Difficulty Estimation and Budget Reallocation

Reinforcement Learning on Pre-Training Data

Understanding Emergent In-Context Learning from a Kernel Regression Perspective

SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMs

DriveQA: Passing the Driving Knowledge Test

Small LLMs Do Not Learn a Generalizable Theory of Mind via Reinforcement Learning

KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding

Lightweight Safety Guardrails via Synthetic Data and RL-guided Adversarial Training

Language-Unlocked ViT (LUViT): Empowering Self-Supervised Vision Transformers with LLMs

Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size

RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering

Product of Experts with LLMs: Boosting Performance on ARC Is a Matter of Perspective

Focus On This, Not That! Steering LLMs with Adaptive Feature Specification

LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective

Agentic Knowledgeable Self-awareness

Each Graph is a New Language: Graph Learning with LLMs

ProgRM: Build Better GUI Agents with Progress Rewards

Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs

Dark LLMs: The Growing Threat of Unaligned AI Models

CLTP: Contrastive Language-Tactile Pre-training for 3D Contact Geometry Understanding

Kimi-Audio Technical Report

DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments

Scaling Laws for Predicting Downstream Performance in LLMs

Fast Training Dataset Attribution via In-Context Learning

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules

Does Training on Synthetic Data Make Models Less Robust?

AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees