语言大模型 / LLM

Trainable Dynamic Mask Sparse Attention

Jingze Shi, Yifan Wu, Yiran Peng, Bingheng Wu, Liangdong Wang, Guang Liu, Yuyu Luo

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 26 pages

Continual Pre-training of MoEs: How robust is your router?

Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar, Ashwinee Panda, Anirban Das, Shi-Xiong Zhang, Stephen Rawls, Sambit Sahu, Eugene Belilovsky, Irina Rish

机构 * Université de Montréal（蒙特利尔大学） ； Mila – Quebec AI Institute（魁北克人工智能研究所） ； Concordia University（康科迪亚大学） ； University of Chicago（芝加哥大学） ； Capital One（Capital One公司） 纠错

专题命中 预训练与数据 ：language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.AI、cs.LG

DRIP: Dynamic patch Reduction via Interpretable Pooling

Yusen Peng, Sachin Kumar

专题命中 预训练与数据 ：language model(abstract)；instruction tuning(abstract)；pretraining(abstract)

Comments Need more refinement

MLLM-Driven Semantic Identifier Generation for Generative Cross-Modal Retrieval

Tianyuan Li, Lei Wang, Ahtamjan Ahmat, Yating Yang, Bo Ma, Rui Dong, Bangju Han

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)

Comments We plan to revise the methodology and update the experimental analysis before resubmission

Towards a Method for Synthetic Generation of Persons with Aphasia Transcripts

Jason M. Pittman, Anton Phillips, Yesenia Medina-Santos, Brielle C. Stark

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages, 1 figure, 7 tables

A Data-Centric Approach to Multilingual E-Commerce Product Search: Case Study on Query-Category and Query-Item Relevance

Yabo Yin, Yang Xi, Jialong Wang, Shanqi Wang, Jiateng Hu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Zero-Shot Vehicle Model Recognition via Text-Based Retrieval-Augmented Generation

Wei-Chia Chang, Yan-Ann Chen

机构 * Yuan Ze University（元智大学） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by The 38th Conference of Open Innovations Association FRUCT, 2025

Watermark Robustness and Radioactivity May Be at Odds in Federated Learning

Leixu Huang, Zedian Shao, Teodora Baluta

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 9 pages, 4 figures (not including citation and appendix) submitted to ICLR 2026

Predicting Task Performance with Context-aware Scaling Laws

Kyle Montgomery, David Park, Jianhong Tu, Michael Bendersky, Beliz Gunel, Dawn Song, Chenguang Wang

机构 * UC Santa Cruz（加州大学圣克ruz分校） ； Washington University in St. Louis（华盛顿大学圣路易斯分校） ； Databricks（Databricks公司） ； Google DeepMind（谷歌DeepMind） ； UC Berkeley（加州大学伯克利分校） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos

Yi Chen, Yuying Ge, Weiliang Tang, Yizhuo Li, Yixiao Ge, Mingyu Ding, Ying Shan, Xihui Liu

机构 * The University of Hong Kong（香港大学） ； ARC Lab, Tencent PCG（腾讯PCG实验室） ； The Chinese University of Hong Kong（香港中文大学） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICCV 2025. Project page: https://chenyi99.github.io/moto/

Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers

Xin Zhao, Xiaojun Chen, Bingshan Liu, Haoyu Gao, Zhendong Zhao, Yilong Chen

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)

Automatic Synthesis of High-Quality Triplet Data for Composed Image Retrieval

Haiwen Li, Delong Liu, Zhaohui Hou, Zhicheng Zhao, Fei Su

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments This paper was originally submitted to ACM MM 2025 on April 12, 2025

VENTURA: Adapting Image Diffusion Models for Unified Task Conditioned Navigation

Arthur Zhang, Xiangyun Meng, Luca Calliari, Dong-Ki Kim, Shayegan Omidshafiei, Joydeep Biswas, Ali Agha, Amirreza Shaban

专题命中 预训练与数据 ：language model(abstract)；foundation model(abstract)；pretraining(abstract)

Comments 9 pages, 6 figures, 3 tables

SpeechWeave: Diverse Multilingual Synthetic Text & Audio Data Generation Pipeline for Training Text to Speech Models

Karan Dua, Puneet Mittal, Ranjeet Gupta, Hitesh Laxmichand Patel

机构 * Oracle AI 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2025

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track) - 2025

Retrieval-augmented GUI Agents with Generative Guidelines

Ran Xu, Kaixin Ma, Wenhao Yu, Hongming Zhang, Joyce C. Ho, Carl Yang, Dong Yu

机构 * Emory University（埃默里大学） ； Tencent AI Lab（腾讯AI实验室） 纠错

专题命中 预训练与数据 ：language model(abstract)；SFT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2025 (Main Conference)

Towards a Comprehensive Scaling Law of Mixture-of-Experts

Guoliang Zhao, Yuhan Fu, Shuaipeng Li, Xingwu Sun, Ruobing Xie, An Wang, Weidong Han, Zhen Yang, Weixuan Sun, Yudong Zhang, Cheng-zhong Xu, Di Wang, Jie Jiang

机构 * Tencent Hunyuan（腾讯文元） ； University of Macau（澳门大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Guided Reality: Generating Visually-Enriched AR Task Guidance with LLMs and Vision Models

Ada Yi Zhao, Aditya Gunturu, Ellen Yi-Luen Do, Ryo Suzuki

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments To appear at UIST 2025

Beyond Language Barriers: Multi-Agent Coordination for Multi-Language Code Generation

Micheline Bénédicte Moumoula, Serge Lionel Nikiema, Albérick Euraste Djire, Abdoul Kader Kabore, Jacques Klein, Tegawendé F. Bissyande

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

DualAlign: Generating Clinically Grounded Synthetic Data

Rumeng Li, Xun Wang, Hong Yu

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Beyond Label Semantics: Language-Guided Action Anatomy for Few-shot Action Recognition

Zefeng Qian, Xincheng Yao, Yifei Huang, Chongyang Zhang, Jiangyong Ying, Hong Sun

机构 * School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与电子工程学院） ； MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能教育部重点实验室） ； The University of Tokyo（东京大学） ； Shanghai AI Laboratory（上海人工智能实验室） ； E-surfing Vision Technology Co., Ltd（亿欧视觉科技有限公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by ICCV2025

ARSP: Automated Repair of Verilog Designs via Semantic Partitioning

Bingkun Yao, Ning Wang, Xiangfeng Liu, Yuxin Du, Yuchen Hu, Hong Gao, Zhe Jiang, Nan Guan

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer

Yushi Lan, Yihang Luo, Fangzhou Hong, Shangchen Zhou, Honghua Chen, Zhaoyang Lyu, Shuai Yang, Bo Dai, Chen Change Loy, Xingang Pan

机构 * S-Lab, Nanyang Technological University, Singapore（南洋理工大学S实验室） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； WICT, Peking University（北京大学WICT学院） ； The University of Hong Kong（香港大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；language model(abstract)；pretraining(abstract)

Comments TL;DR: Streaming 4D reconstruction using causal transformer. Project page: https://nirvanalan.github.io/projects/stream3r

TBAC-UniImage: Unified Understanding and Generation by Ladder-Side Diffusion Tuning

Junzhe Xu, Yuyang Yin, Xi Chen

机构 * Basic Algorithm Center, PCG, Tencent（腾讯基础算法中心、PCG、腾讯） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)

3DFroMLLM: 3D Prototype Generation only from Pretrained Multimodal LLMs

Noor Ahmed, Cameron Braunstein, Steffen Eger, Eddy Ilg

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)

MinionsLLM: a Task-adaptive Framework For The Training and Control of Multi-Agent Systems Through Natural Language

Andres Garcia Rincon, Eliseo Ferrante

机构 * Vrije Universiteit Amsterdam, The Netherlands（阿姆斯特丹自由大学，荷兰） ； NYU Abu Dhabi, United Arab Emirates（纽约大学阿布扎克分校，阿联酋） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

SmartLLMs Scheduler: A Framework for Cost-Effective LLMs Utilization

Yueyue Liu, Hongyu Zhang, Yuantian Miao

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Vulnerability Handling of AI-Generated Code -- Existing Solutions and Open Challenges

Sabrina Kaniewski, Dieter Holstein, Fabian Schmidt, Tobias Heer

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted for publication @ IEEE AIxSET 2024; 4 pages, 2 Figures

Can We End the Cat-and-Mouse Game? Simulating Self-Evolving Phishing Attacks with LLMs and Genetic Algorithms

Seiji Sato, Tetsushi Ohki, Masakatsu Nishigaki

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling

Chenyu Huang, Zhengyang Tang, Shixi Hu, Ruoqing Jiang, Xin Zheng, Dongdong Ge, Benyou Wang, Zizhuo Wang

机构 * Shanghai University of Finance and Economics（上海财经大学） ； The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳）） ； Shenzhen Research Institute of Big Data（深圳大数据研究院） ； Shanghai Jiao Tong University（上海交通大学） ； Cardinal Operations ； Columbia University（哥伦比亚大学） ； Duke University（杜克大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted by Operations Research

Journal ref Operations Research (2025), published online ahead of print

EventVAD: Training-Free Event-Aware Video Anomaly Detection

Yihua Shao, Haojin He, Sijie Li, Siyu Chen, Xinwei Long, Fanhu Zeng, Yuxuan Fan, Muyang Zhang, Ziyang Yan, Ao Ma, Xiaochen Wang, Hao Tang, Yan Wang, Shuyan Li

机构 * Peking University（北京大学） ； Guangdong University of Technology（广东工业大学） ； The University of Sheffield（谢菲尔德大学） ； University of Science and Technology Beijing（北京科技大学） ； Tsinghua University（清华大学） ； The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） ； Nanjing University（南京大学） ； University of Trento（特伦特大学） ； Queen's University Belfast（贝尔法斯特女王大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Paper was accepted by ACM MM 2025; Code: https://github.com/YihuaJerry/EventVAD