语言大模型 / LLM

SinLlama -- A Large Language Model for Sinhala

H. W. K. Aravinda, Rashad Sirajudeen, Samith Karunathilake, Nisansa de Silva, Surangika Ranathunga, Rishemjit Kaur

机构 * Central Scientific Instruments Organisation, Academy of Scientific and Innovative Research（中央科学仪器组织，科学与创新研究院） ； School of Mathematical and Computational Sciences, Massey University（数学与计算科学学院，梅西大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL

Is Training Data Quality or Quantity More Impactful to Small Language Model Performance?

Aryan Sajith, Krishna Chaitanya Rao Kathala

专题命中 预训练与数据 ：language model(title,abstract)；small language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 14 pages, 5 tables, 4 figures | Accepted at International Conference on Neural Computing for Advanced Applications 2025, Conference info: https://aaci.org.hk/ncaa2025

Journal ref Commun. Comput. Inf. Sci. 2665 (2025) 75-87

Uncovering Pretraining Code in LLMs: A Syntax-Aware Attribution Approach

Yuanheng Li, Zhuoyang Chen, Xiaoyun Liu, Yuhao Wang, Mingwei Liu, Yang Shi, Kaifeng Huang, Shengjie Zhao

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Paper has been accepted by AAAI 2026

Rethinking Text-based Protein Understanding: Retrieval or LLM?

Juntong Wu, Zijing Liu, He Cao, Hao Li, Bin Feng, Zishan Shu, Ke Yu, Li Yuan, Yu Li

机构 * Peking University, Shenzhen Graduate School（北京大学深圳研究生院） ； International Digital Economy Academy (IDEA)（国际数字经济学院） 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted by Empirical Methods in Natural Language Processing 2025 (EMNLP 2025) Main Conference

AI Brown and AI Koditex: LLM-Generated Corpora Comparable to Traditional Corpora of English and Czech Texts

Jiří Milička, Anna Marklová, Václav Cvrček

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws

Zhixuan Pan, Shaowen Wang, Jian Li

机构 * Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院） ； School of Computer Science and Engineering, Beihang University（北航计算机科学与工程学院） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Tiny language models

Ronit D. Gross, Yarden Tzach, Tal Halevi, Ella Koresh, Ido Kanter

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL

Comments 23 pages, 1 figure and 12 tables, The data and code that support the findings of this study are openly available in a GitHub repository at https://github.com/Rg32601/Tiny-Language-Models

Journal ref Physica A: Statistical Mechanics and its Applications Volume 681, 1 January 2026, 131102

HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor

Zihui Wu, Haichang Gao, Jiacheng Luo, Zhaoxiang Liu

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

LLM Teacher-Student Framework for Text Classification With No Manually Annotated Data: A Case Study in IPTC News Topic Classification

Taja Kuzman, Nikola Ljubešić

机构 * Jožef Stefan International Postgraduate School（乔塞夫·斯塔芬国际研究生学校） ； University of Ljubljana（卢布尔雅那大学） 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments This work has been accepted and published in the IEEE Access journal. This arXiv version is retained for archival purposes. Readers should use and cite the IEEE Access Version available at https://ieeexplore.ieee.org/document/10900365

Journal ref IEEE Access 2025

Who Is the Story About? Protagonist Entity Recognition in News

Jorge Gabín, M. Eduardo Ares, Javier Parapar

机构 * Linknovate Science（Linknovate科学） ； IRLab CITIC Computer Science Department University of A Coruña（IRLab CITIC计算机科学系大学） ； University of A Coruña（阿拉斯卡大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction

Zhonghang Li, Long Xia, Lei Shi, Yong Xu, Dawei Yin, Chao Huang

机构 * The University of Hong Kong（香港大学） ； South China University of Technology（华南理工大学） ； Baidu Inc.（百度公司） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments 12 pages

Journal ref ACM TIST, 2025

Retracing the Past: LLMs Emit Training Data When They Get Lost

Myeongseob Ko, Nikhil Reddy Billa, Adam Nguyen, Charles Fleming, Ming Jin, Ruoxi Jia

机构 * Virginia Tech（弗吉尼亚理工大学） ； Cisco Research（思科研究） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；SFT(abstract)；分类 cs.CL、cs.AI

Comments The 2025 Conference on Empirical Methods in Natural Language Processing

Describe Where You Are: Improving Noise-Robustness for Speech Emotion Recognition with Text Description of the Environment

Seong-Gyun Leem, Daniel Fulford, Jukka-Pekka Onnela, David Gard, Carlos Busso

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Future of AI Models: A Computational perspective on Model collapse

Trivikram Satharasi, S Sitharama Iyengar

机构 * University of Florida（佛罗里达大学） ； Florida International University（佛罗里达国际大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Submitted to Springer Nature. Code Available at https://github.com/t-satharasi/AI-Modal-Collapse-Code-for-Reproduction.git

Continual Pre-training of MoEs: How robust is your router?

Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar, Ashwinee Panda, Anirban Das, Shi-Xiong Zhang, Stephen Rawls, Sambit Sahu, Eugene Belilovsky, Irina Rish

机构 * Université de Montréal（蒙特利尔大学） ； Mila – Quebec AI Institute（魁北克人工智能研究所） ； Concordia University（康科迪亚大学） ； University of Chicago（芝加哥大学） ； Capital One（Capital One公司） 纠错

专题命中 预训练与数据 ：language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.AI、cs.LG

Wasm: A Pipeline for Constructing Structured Arabic Interleaved Multimodal Corpora

Khalil Hennara, Ahmad Bastati, Muhammad Hreden, Mohamed Motasim Hamed, Zeina Aldallal, Sara Chrouf, Safwan AlModhayan

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Scaling Laws and In-Context Learning: A Unified Theoretical Framework

Sushant Mehta, Ishan Gupta

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Journal ref Workshop on Principles of Generative Modeling (PriGM) @ EurIPS2025

GauSSmart: Enhanced 3D Reconstruction through 2D Foundation Models and Geometric Filtering

Alexander Valverde, Brian Xu, Yuyin Zhou, Meng Xu, Hongyun Wang

机构 * University of California, Santa Cruz（加州大学圣克鲁兹分校） ； Brown University（布朗大学） ； Kean University（凯恩大学） 纠错

专题命中 预训练与数据 ：foundation model(title)

LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corpora

Luyao Zhuang, Shengyuan Chen, Yilin Xiao, Huachi Zhou, Yujing Zhang, Hao Chen, Qinggang Zhang, Xiao Huang

机构 * The Department of Computing, Hong Kong Polytechnic University, Hong Kong SAR（香港理工大学计算机系） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL

Does TabPFN Understand Causal Structures?

Omar Swelam, Lennart Purucker, Jake Robertson, Hanne Raum, Joschka Boedecker, Frank Hutter

机构 * University of Freiburg（弗赖堡大学） ； Prior Labs（Prior实验室） ； ELLIS Institute Tübingen（图宾根ELLIS研究所） 纠错

专题命中 预训练与数据 ：foundation model(abstract)；分类 cs.LG

DRIVE: Data Curation Best Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generation

Speed Zhu, Jianwei Cai, Guang Chen, Lulu Wu, Saiyong Yang, Wiggin Zhou

机构 * Hunyuan Team, Tencent（腾讯 Hunyuan 团队） 纠错

专题命中 预训练与数据 ：SFT(abstract)；分类 cs.LG

Comments 15 pages, 8 figures

CoMA: Complementary Masking and Hierarchical Dynamic Multi-Window Self-Attention in a Unified Pre-training Framework

Jiaxuan Li, Qing Xu, Xiangjian He, Ziyu Liu, Chang Xing, Zhen Chen, Daokun Zhang, Rong Qu, Chang Wen Chen

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI

Comments 9 pages, 5 figures

Attention on flow control: transformer-based reinforcement learning for lift regulation in highly disturbed flows

Zhecheng Liu, Jeff D. Eldredge

机构 * Mechanical and Aerospace Engineering, University of California, Los Angeles, CA 90095-1597, USA（机械与航空航天工程系，加州大学洛杉矶分校） 纠错

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.LG

Identity Card Presentation Attack Detection: A Systematic Review

Esteban M. Ruiz, Juan E. Tapia, Reinel T. Soto, Christoph Busch

机构 * da/sec-Biometrics and Internet Security Research Group, Hochschule Darmstadt（da/sec生物识别与互联网安全研究组，达姆施塔特高等学院） ； Universidad Autónoma de Manizales（自动化工学院） ； Universidad de Caldas（卡利斯大学） 纠错

专题命中 预训练与数据 ：foundation model(abstract)

STARS: Self-supervised Tuning for 3D Action Recognition in Skeleton Sequences

Soroush Mehraban, Mohammad Javad Rajabi, Andrea Iaboni, Babak Taati

机构 * University of Toronto（多伦多大学） ； Vector Institute（向量研究所） ； KITE Research Institute（KITE研究机构） 纠错

专题命中 预训练与数据 ：pretraining(abstract)

Oblivionis: A Lightweight Learning and Unlearning Framework for Federated Large Language Models

Fuyao Zhang, Xinyu Yan, Tiantong Wu, Wenjie Li, Tianxiang Chen, Yang Cao, Ran Yan, Longtao Huang, Wei Yang Bryan Lim, Qiang Yang

专题命中 指令微调 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Importance-Aware Data Selection for Efficient LLM Instruction Tuning

Tingyu Jiang, Shen Li, Yiyao Song, Lan Zhang, Hualei Zhu, Yuan Zhao, Xiaohang Xu, Kenjiro Taura, Hao Henry Wang

专题命中 指令微调 ：LLM(title,abstract)；instruction tuning(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by AAAI 2026 Oral

Sensitivity of Small Language Models to Fine-tuning Data Contamination

Nicy Scaria, Silvester John Joseph Kennedy, Deepak Subramani

专题命中 指令微调 ：language model(title,abstract)；small language model(title,abstract)；instruction tuning(abstract)；分类 cs.CL、cs.AI

Lived Experience in Dialogue: Co-designing Personalization in Large Language Models to Support Youth Mental Well-being

Kathleen W. Guan, Sarthak Giri, Mohammed Amara, Bernard J. Jansen, Enrico Liscio, Milena Esherick, Mohammed Al Owayyed, Ausrine Ratkute, Gayane Sedrakyan, Mark de Reuver, Joao Fernando Ferreira Goncalves, Caroline A. Figueroa

专题命中 指令微调 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.AI

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective

Heshan Fernando, Han Shen, Parikshit Ram, Yi Zhou, Horst Samulowitz, Nathalie Baracaldo, Tianyi Chen

机构 * Rensselaer Polytechnic Institute（罗切斯特理工学院） ； IBM Research（IBM研究院） ； Cornell University（康奈尔大学） 纠错

专题命中 指令微调 ：LLM(title,abstract)；post-training(abstract)；SFT(abstract)；RLHF(abstract)