语言大模型 / LLM

NeuroTTT: Bridging Pretraining-Downstream Task Misalignment in EEG Foundation Models via Test-Time Training

Suli Wang, Yangshen Deng, Zhenghua Bao, Xinyu Zhan, Yiqun Duan

机构 * Technical University of Darmstadt（达姆斯塔特技术大学） ； University of Edinburgh（爱丁堡大学） ； University of Technology Sydney（悉尼技术大学） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)；分类 cs.LG

Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls

Feiyang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad, Mostafa Elhoushi, Shubhabrata Sengupta, Shang-Wen Li, Ramya Raghavendra, Ruoxi Jia, Carole-Jean Wu

机构 * FAIR at Meta（Meta 的 FAIR） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a Main Conference paper at EMNLP 2025

Enhancing Transformer-Based Rerankers with Synthetic Data and LLM-Based Supervision

Dimitar Peshevski, Kiril Blazhevski, Martin Popovski, Gjorgji Madjarov

机构 * Faculty of Computer Science and Engineering（计算机科学与工程学院） ； Machine Learning Department（机器学习系） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by RANLP 2025

LLM-Based Multi-Task Bangla Hate Speech Detection: Type, Severity, and Target

Md Arid Hasan, Firoj Alam, Md Fahad Hossain, Usman Naseem, Syed Ishtiaque Ahmed

机构 * University of Toronto（多伦多大学） ； Qatar Computing Research Institute（卡塔尔计算研究院） ； Daffodil International University（达芙妮国际大学） ； Macquarie University（麦考瑞大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；pretraining(abstract)；prompting(abstract)；分类 cs.CL

Scaling Laws for Optimal Data Mixtures

Mustafa Shukor, Louis Bethune, Dan Busbridge, David Grangier, Enrico Fini, Alaaeldin El-Nouby, Pierre Ablin

机构 * Sorbonne University（索邦大学） ； Apple（苹果公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?

Hanqun Cao, Hongrui Zhang, Junde Xu, Zhou Zhang, Lingdong Shen, Minghao Sun, Ge Liu, Jinbo Xu, Wu-Jun Li, Jinren Ni, Cesar de la Fuente-Nunez, Tianfan Fu, Yejin Choi, Pheng-Ann Heng, Fang Wu

机构 * The Chinese University of Hong Kong（香港中文大学） ； Peking University（北京大学） ； Stanford University（斯坦福大学） ； Nanjing University（南京大学） ； University of Pennsylvania（宾夕法尼亚大学） ； National University of Singapore（新加坡国立大学） ； University of Illinois Urbana-Champaign（伊利诺伊大学香槟分校） ； Toyota Technological Institute at Chicago（芝加哥丰田技术研究所） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 24 pages, 7 figures, 4 tables

Out-of-Distribution Detection using Synthetic Data Generation

Momin Abbas, Muneeza Azmat, Raya Horesh, Mikhail Yurochkin

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Comments Accepted to COLM 2025. Camera-ready version

F2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source Data

Ziyin Zhang, Zihan Liao, Hang Yu, Peng Di, Rui Wang

机构 * Ant Group（蚂蚁集团） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；pretraining(abstract)

VENTURA: Adapting Image Diffusion Models for Unified Task Conditioned Navigation

Arthur Zhang, Xiangyun Meng, Luca Calliari, Dong-Ki Kim, Shayegan Omidshafiei, Joydeep Biswas, Ali Agha, Amirreza Shaban

专题命中 预训练与数据 ：language model(abstract)；foundation model(abstract)；pretraining(abstract)

Comments 9 pages, 6 figures, 3 tables

SpeechWeave: Diverse Multilingual Synthetic Text & Audio Data Generation Pipeline for Training Text to Speech Models

Karan Dua, Puneet Mittal, Ranjeet Gupta, Hitesh Laxmichand Patel

机构 * Oracle AI 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2025

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track) - 2025

Study on LLMs for Promptagator-Style Dense Retriever Training

Daniel Gwon, Nour Jedidi, Jimmy Lin

机构 * Massachusetts Institute of Technology Lincoln Laboratory（麻省理工学院林肯实验室） ； University of Waterloo（滑铁卢大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments CIKM 2025 short research paper

Exploring Database Normalization Effects on SQL Generation

Ryosuke Kohita

机构 * CyberAgent 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted to CIKM 2025

Growing Visual Generative Capacity for Pre-Trained MLLMs

Hanyu Wang, Jiaming Han, Ziyan Yang, Qi Zhao, Shanchuan Lin, Xiangyu Yue, Abhinav Shrivastava, Zhenheng Yang, Hao Chen

机构 * University of Maryland, College Park（马里兰大学学院公园分校） ； CUHK MMLab（香港大学MMLab） ； ByteDance（字节跳动） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.LG

Comments Project page: https://hywang66.github.io/bridge/

SLAP: Learning Speaker and Health-Related Representations from Natural Language Supervision

Angelika Ando, Auguste Crabeil, Adrien Lesage, Rachid Riad

专题命中 预训练与数据 ：foundation model(abstract)；pretraining(abstract)

AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs

Feiyang Kang, Yifan Sun, Bingbing Wen, Si Chen, Dawn Song, Rafid Mahmood, Ruoxi Jia

机构 * Virginia Tech（弗吉尼亚理工大学） ； UIUC（伊利诺伊大学香槟分校） ； University of Washington（华盛顿大学） ； UC Berkeley（加州大学伯克利分校） ； University of Ottawa（渥太华大学） ； NVIDIA（英伟达） 纠错

专题命中 预训练与数据 ：LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a conference paper at COLM 2025

Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective

Wen Yang, Junhong Wu, Chong Li, Chengqing Zong, Jiajun Zhang

机构 * School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） 纠错

专题命中 预训练与数据 ：post-training(abstract)；分类 cs.CL、cs.AI

Comments Work in progress

Meta-Transfer Derm-Diagnosis: Exploring Few-Shot Learning and Transfer Learning for Skin Disease Classification in Long-Tail Distribution

Zeynep Özdemir, Hacer Yalim Keles, Ömer Özgür Tanrıöver

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.LG

Comments This is the accepted version of the article to appear in IEEE Journal of Biomedical and Health Informatics. DOI: 10.1109/JBHI.2025.3615479

Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology

Saghir Alfasly, Wataru Uegami, MD Enamul Hoq, Ghazal Alabtah, H. R. Tizhoosh

专题命中 预训练与数据 ：foundation model(abstract)

Comments NeurIPS 2025

RS-OOD: A Vision-Language Augmented Framework for Out-of-Distribution Detection in Remote Sensing

Chenhao Wang, Yingrui Ji, Yu Meng, Yunjian Zhang, Yao Zhu

专题命中 预训练与数据 ：language model(abstract)

UpSafe$^\circ$C: Upcycling for Controllable Safety in Large Language Models

Yuhao Sun, Zhuoer Xu, Shiwen Cui, Kun Yang, Lingyun Yu, Yongdong Zhang, Hongtao Xie

专题命中 指令微调 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Investigating ReLoRA: Effects on the Learning Dynamics of Small Language Models

Yuval Weiss, David Demitri Africa, Paula Buttery, Richard Diehl Martinez

机构 * University of Cambridge（剑桥大学） 纠错

专题命中 指令微调 ：language model(title,abstract)；small language model(title,abstract)；LLM(abstract)；large language model(abstract)

Comments 12 Pages, 6 Tables, 8 Figures

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead

Feiyang Kang, Michael Kuchnik, Karthik Padthe, Marin Vlastelica, Ruoxi Jia, Carole-Jean Wu, Newsha Ardalani

机构 * FAIR at Meta（Meta 的 FAIR） ； Meta 纠错

专题命中 指令微调 ：post-training(title,abstract)；SFT(title,abstract)；large language model(abstract)；language model(abstract)

Comments Preprint. Under Review

Bridging Collaborative Filtering and Large Language Models with Dynamic Alignment, Multimodal Fusion and Evidence-grounded Explanations

Bo Ma, LuYao Liu, Simon Lau, Chandler Yuan, and XueY Cui, Rosie Zhang

机构 * Department of Software \& Microelectronics, Peking University, Beijing, China ； Economic Law School, China University of Political Science ； Financial Media, Peking University, ChangSha, China 纠错

专题命中 指令微调 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

OR-Toolformer: Modeling and Solving Operations Research Problems with Tool Augmented Large Language Models

Jianzhang Zhang, Jialong Zhou, Chuang Liu

机构 * Alibaba Business School（阿里巴巴商业学院） ； Hangzhou Normal University（杭州师范大学） 纠错

专题命中 指令微调 ：large language model(title,abstract)；language model(title,abstract)；分类 cs.AI、cs.LG

Efficient Training of Robust Traditional Chinese LLaMA-1B on a Single Consumer GPU: Continual Pre-training, SFT, and DPO

Yu-Cheng Chih, Ming-Tao Duan, Yong-Hao Hou

专题命中 指令微调 ：SFT(title,abstract)；language model(abstract,comments)；small language model(abstract,comments)；preference optimization(abstract)

Comments 17 pages, 1 figures, 2 tables. Technical report. Introduces PureTC-1B, an adapter-based pipeline for stabilizing Small Language Models in Traditional Chinese using CPT, SFT, and DPO

Fine-tuning with RAG for Improving LLM Learning of New Skills

Humaid Ibrahim, Nikolai Rozanov, Marek Rei

专题命中 指令微调 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Under review at ICLR 2026

Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment Trajectories

Nilay Naharas, Dang Nguyen, Nesihan Bulut, Mohammadhossein Bateni, Vahab Mirrokni, Baharan Mirzasoleiman

机构 * Department of Computer Science, University of California Los Angeles（加州大学洛杉矶分校计算机科学系） ； Google Research（谷歌研究） 纠错

专题命中 指令微调 ：language model(title,abstract)；large language model(abstract)；instruction tuning(abstract)；分类 cs.LG

Comments 30 pages, 10 figures, 5 tables, link: https://bigml-cs-ucla.github.io/XMAS-project-page/

LLM Based Sentiment Classification From Bangladesh E-Commerce Reviews

Sumaiya Tabassum

专题命中 指令微调 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs

Yongyi Su, Haojie Zhang, Shijie Li, Nanqing Liu, Jingyi Liao, Junyi Pan, Yuan Liu, Xiaofen Xing, Chong Sun, Chen Li, Nancy F. Chen, Shuicheng Yan, Xulei Yang, Xun Xu

机构 * South China University of Technology（华南理工大学） ； Institute for Infocomm Research (I 2 R), A*STAR（信息通信研究所（I 2 R），A*STAR） ； WeChat Vision, Tencent Inc.（微信视觉，腾讯公司） ； Foshan University（佛山大学） ； Nanyang Technological University（南洋理工大学） ； National University of Singapore（新加坡国立大学） 纠错

专题命中 指令微调 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 24 pages, 12 figures and 9 tables

Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning

Kaustubh Ponkshe, Raghav Singhal, Eduard Gorbunov, Alexey Tumanov, Samuel Horvath, Praneeth Vepakomma

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） ； Georgia Institute of Technology（佐治亚理工学院） ； Massachusetts Institute of Technology（麻省理工学院） 纠错

专题命中 指令微调 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Kaustubh Ponkshe and Raghav Singhal contributed equally to this work