语言大模型 / LLM

Learning Dynamics of Meta-Learning in Small Model Pretraining

David Demitri Africa, Yuval Weiss, Paula Buttery, Richard Diehl Martinez

机构 * University of Cambridge（剑桥大学） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；small language model(abstract)

Comments Accepted (oral) to Student Research Workshop at IJCNLP-AACL 2025

ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining

Xincheng Yao, Yan Luo, Zefeng Qian, Chongyang Zhang

机构 * School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与电子工程学院） ； MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能MOE重点实验室） ； College of Artificial Intelligence, Nanjing Agricultural University（南京农业大学人工智能学院） ； Key Laboratory of Livestock Farming Equipment, Ministry of Agriculture and Rural Affairs, Nanjing Agricultural University（南京农业大学农业机械化关键实验室） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)

Comments Accepted by NeurIPS 2025

Preserving Speaker Information in Direct Speech-to-Speech Translation with Non-Autoregressive Generation and Pretraining

Rui Zhou, Akinori Ito, Takashi Nose

机构 * Graduate School of Engineering, Tohoku University（东大理工大学研究生院） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)

ManufactuBERT: Efficient Continual Pretraining for Manufacturing

Robin Armingaud, Romaric Besançon

专题命中 预训练与数据 ：pretraining(title)；分类 cs.CL

Comments Submitted to LREC 2026

LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence

Xingxuan Zhang, Gang Ren, Han Yu, Hao Yuan, Hui Wang, Jiansheng Li, Jiayun Wu, Lang Mo, Li Mao, Mingchao Hao, Ningbo Dai, Renzhe Xu, Shuyang Li, Tianyang Zhang, Yue He, Yuanrui Wang, Yunjia Zhang, Zijing Xu, Dongzhe Li, Fang Gao, Hao Zou, Jiandong Liu, Jiashuo Liu, Jiawei Xu, Kaijie Cheng, Kehan Li, Linjun Zhou, Qing Li, Shaohua Fan, Xiaoyu Lin, Xinyan Han, Xuanyue Li, Yan Lu, Yuan Xue, Yuanyuan Jiang, Zimu Wang, Zhenlei Wang, Peng Cui

机构 * Stable AI & Tsinghua University（Stable AI 与 清华大学） 纠错

专题命中 预训练与数据 ：foundation model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 61 pages

Role-SynthCLIP: A Role Play Driven Diverse Synthetic Data Approach

Yuanxiang Huangfu, Chaochao Wang, Weilei Wang

机构 * PatSnap Co., LTD.（PatSnap公司） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)

What Can String Probability Tell Us About Grammaticality?

Jennifer Hu, Ethan Gotlieb Wilcox, Siyuan Song, Kyle Mahowald, Roger P. Levy

机构 * Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University（哈佛大学自然与人工智能研究 institute） 纠错

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

When Data Falls Short: Grokking Below the Critical Threshold

Vaibhav Singh, Eugene Belilovsky, Rahaf Aljundi

机构 * Concordia University（康科迪亚大学） ； Mila ； Toyota Motor Europe（丰田欧洲公司） 纠错

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.LG

Comments 6 pages