Should We Still Pretrain Encoders with Masked Language Modeling?
我们还应该用掩码语言模型预训练编码器吗?
Hippolyte Gisserot-Boukhlef, Nicolas Boizard, Manuel Faysse, Duarte M. Alves, Emmanuel Malherbe, André F. T. Martins, Céline Hudelot, Pierre Colombo
机构
*
Artefact Research Center(Artefact 研究中心)
;
Diabolocom
;
TransPerfect
;
Cohere
;
MICS, CentraleSupélec, Université Paris-Saclay(MICS,CentraleSupélec,巴黎萨克雷大学)
;
Instituto de Telecomunicações(电信研究所)
;
Instituto Superior Técnico & Universidade de Lisboa (Lisbon ELLIS Unit)(里斯本大学(里斯本 ELLIS 单位))
SpecAware: A Spectral-Content Aware Foundation Model for Unifying Multi-Sensor Learning in Hyperspectral Remote Sensing Mapping
SpecAware: 一种面向光谱-内容的统一多传感器学习的超光谱遥感制图基础模型
Renjie Ji, Xue Wang, Chao Niu, Wen Zhang, Yong Mei, Kun Tan
机构
*
Key Laboratory of Spatial-Temporal Big Data Analysis and Application of Natural Resources in Megacities (Ministry of Natural Resources), East China Normal University(空间-时间大数据分析与应用国家级重点实验室(自然资源部),东华师范大学)
;
School of Geospatial Artificial Intelligence, East China Normal University(地理信息人工智能学院,东华师范大学)
;
Key Laboratory of Geographic Information Science (Ministry of Education), East China Normal University(地理信息科学重点实验室(教育部),东华师范大学)
;
School of Geographic Sciences, East China Normal University(地理科学学院,东华师范大学)
;
Shanghai Municipal Institute of Surveying and Mapping(上海市测绘院)
;
Institute of Defense Engineering, AMS(工程院,AMS)
BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop
BabyLM四周年暨多语言研究:2026年BabyLM研讨会征文通知
Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Jaap Jumelet, Tal Linzen, Aaron Mueller, Suchir Salhan, Raj Sanjay Shah, Alex Warstadt, Ethan Gotlieb Wilcox
机构
*
IBM Research(IBM研究院)
;
MIT(麻省理工学院)
;
ETH Zürich(苏黎世联邦理工学院)
;
Cornell University(康奈尔大学)
;
University of Groningen(格罗宁根大学)
;
NYU(纽约大学)
;
Boston University(波士顿大学)
;
University of Cambridge(剑桥大学)
;
Georgia Tech(佐治亚理工学院)
;
UC San Diego(南加州大学)
;
Georgetown University(乔治城大学)
机构
*
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院)
;
Shanghai Engineering Research Center of Intelligent Vision and Imaging(智能视觉与成像上海工程研究中心)
;
Tongyi Lab, Alibaba Group(阿里集团通义实验室)
Transfer Learning in Infinite Width Feature Learning Networks
在无限宽度特征学习网络中进行迁移学习
Clarissa Lauditi, Blake Bordelon, Cengiz Pehlevan
机构
*
John A. Paulson School of Engineering and Applied Sciences(约翰·A·保罗森工程与应用科学学院)
;
Center for Mathematical Sciences and Applications(数学科学与应用中心)
;
Kempner Institute for the Study of Natural and Artificial Intelligence(自然与人工智能研究 institute)
;
Harvard University(哈佛大学)
机构
*
Zhejiang University(浙江大学)
;
Binjiang Institute of Zhejiang University(浙江大学滨江学院)
;
Om AI Research(奥姆人工智能研究)
;
Stanford University(斯坦福大学)
;
ETH Zürich(苏黎世联邦理工学院)
专题命中
指令微调
:large language model(title,abstract);language model(title,abstract);LLM(abstract);SFT(abstract)
A Survey on Federated Fine-tuning of Large Language Models
大型语言模型联邦微调综述
Yebo Wu, Chunlin Tian, Jingguang Li, He Sun, Kahou Tam, Zhanting Zhou, Haicheng Liao, Jing Xiong, Zhijiang Guo, Li Li, Chengzhong Xu
机构
*
State Key Laboratory of IOTSC, University of Macau(物联网科学与技术国家重点实验室,澳门大学)
;
University of Electronic Science and Technology of China(电子科技大学)
;
The University of Hong Kong(香港大学)
;
Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州))
专题命中
指令微调
:large language model(title,abstract);language model(title,abstract);LLM(abstract);分类 cs.LG
AI总结
本文综述了大型语言模型联邦微调的技术、挑战与应用,旨在为隐私保护AI的发展提供指导。
CommentsAccepted by Transactions on Machine Learning Research (TMLR), 2026
Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models
递归自聚合解锁大语言模型的深度思考
Siddarth Venkatraman, Vineet Jain, Sarthak Mittal, Vedant Shah, Johan Obando-Ceron, Yoshua Bengio, Brian R. Bartoldson, Bhavya Kailkhura, Guillaume Lajoie, Glen Berseth, Nikolay Malkin, Moksh Jain
机构
*
Mila – Québec AI Institute(魁北克AI研究所)
;
Université de Montréal(蒙特利尔大学)
;
McGill University(麦吉尔大学)
;
LawZero(法零)
;
LLNL(劳伦斯利弗莫尔国家实验室)
;
University of Edinburgh(爱丁堡大学)
;
CIFAR AI Chair(CIFAR人工智能 chair)
;
CIFAR Fellow
专题命中
指令微调
:large language model(title,abstract);language model(title,abstract);分类 cs.LG;LLM(comments)
Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA
通过稀疏和正交LoRA实现无线联邦多任务大语言模型微调
Nuocheng Yang, Sihua Wang, Ouwen Huan, Mingzhe Chen, Tony Q. S. Quek, Changchuan Yin
机构
*
Beijing Laboratory of Advanced Information Network(北京先进信息网络实验室)
;
Beijing Key Laboratory of Network System Architecture and Convergence(北京网络系统架构与融合重点实验室)
;
Beijing University of Posts and Telecommunications(北京邮电大学)
;
Department of Electrical and Computer Engineering and Institute for Data Science and Computing(电气与计算机工程系和数据科学与计算研究所)
;
University of Miami(迈阿密大学)
;
Information Systems Technology and Design Pillar(信息系统技术与设计支柱)
专题命中
指令微调
:LLM(title,abstract);large language model(abstract);language model(abstract);prompting(abstract)
机构
*
Zhejiang University(浙江大学)
;
Westlake University(西湖大学)
;
Department of Psychology, Jing Hengyi School of Education, Hangzhou Normal University(心理系,金恒毅教育学院,杭州师范大学)
专题命中
指令微调
:large language model(title);language model(title);LLM(abstract);分类 cs.CL
Silent Inconsistency in Data-Parallel Full Fine-Tuning: Diagnosing Worker-Level Optimization Misalignment
数据并行全微调中的沉默不一致:诊断工作级别优化不一致
Hong Li, Zhen Zhou, Honggang Zhang, Yuping Luo, Xinyue Wang, Han Gong, Zhiyuan Liu
机构
*
School of Transportation Southeast University(交通学院东南大学)
;
Department of Logistics and Maritime Studies The Hong Kong Polytechnic University(物流与海运研究部香港理工大学)
专题命中
指令微调
:large language model(abstract);language model(abstract);分类 cs.AI、cs.LG
CryoLVM: Self-supervised Learning from Cryo-EM Density Maps with Large Vision Models
CryoLVM: 从冷冻电镜密度图中利用大视觉模型进行自监督学习
Weining Fu, Kai Shu, Kui Xu, Qiangfeng Cliff Zhang
机构
*
State Key Laboratory of Membrane Biology, Beijing Frontier Research Center of Biological Structures, Tsinghua-Peking Joint Center for Life Sciences, School of Life Sciences, Tsinghua University, Beijing, China(膜生物学国家重点实验室,北京生物结构前沿研究中心,清华-北大联合生命科学中心,生命科学学院,清华大学,北京,中国)
;
State Key Laboratory of Membrane Biology-Membrane Structure and Artificial Intelligence Biology Branch,Hangzhou, China(膜生物学国家重点实验室-膜结构与人工智能生物学分支,杭州,中国)
;
State Key Laboratory of Membrane Biology, Beijing Tsinghua Institute for Frontier Interdisciplinary Innovation, Beijing, China(膜生物学国家重点实验室,北京清华大学前沿交叉创新研究院,北京,中国)
Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
Actor-Curator: 通过策略改进带状机为RL后训练实现联合适应课程学习
Zhengyao Gu, Jonathan Light, Raul Astudillo, Ziyu Ye, Langzhou He, Henry Peng Zou, Wei Cheng, Santiago Paternain, Philip S. Yu, Yisong Yue
机构
*
University of Illinois Chicago(伊利诺伊大学香槟分校)
;
Caltech(加州理工学院)
;
RPI(罗切斯特理工学院)
;
MBZUAI(澳门大学人工智能研究院)
;
University of Chicago(芝加哥大学)
;
NEC Laboratories America(NEC美国实验室)
专题命中
后训练与偏好优化
:post-training(title,abstract);LLM(abstract);large language model(abstract);language model(abstract)