语言大模型 / LLM

TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision

Yunyi Zhang, Ruozhen Yang, Xueqiang Xu, Rui Li, Jinfeng Xiao, Jiaming Shen, Jiawei Han

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； University of Science and Technology of China（中国科学技术大学） ； Google Deepmind（谷歌DeepMind） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Accepted to WWW 2025 Research Track

BTS: Harmonizing Specialized Experts into a Generalist LLM

Qizhen Zhang, Prajjwal Bhargava, Chloe Bi, Chris X. Cai, Jakob Foerster, Jeremy Fu, Punit Singh Koura, Ruan Silva, Sheng Shen, Emily Dinan, Suchin Gururangan, Mike Lewis

机构 * GenAI at Meta（Meta GenAI团队） ； University of Oxford（牛津大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs

Zixuan Hu, Yongxian Wei, Li Shen, Chun Yuan, Dacheng Tao

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Crystal: Illuminating LLM Abilities on Language and Code

Tianhua Tao, Junbo Li, Bowen Tan, Hongyi Wang, William Marshall, Bhargav M Kanakiya, Joel Hestness, Natalia Vassilieva, Zhiqiang Shen, Eric P. Xing, Zhengzhong Liu

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Published as a conference paper at COLM 2024

Language Agents Meet Causality -- Bridging LLMs and Causal World Models

John Gkountouras, Matthias Lindemann, Phillip Lippe, Efstratios Gavves, Ivan Titov

专题命中 预训练与数据 ：language agent(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Project page: https://j0hngou.github.io/LLMCWM/

OpenGraph: Towards Open Graph Foundation Models

Lianghao Xia, Ben Kao, Chao Huang

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by EMNLP'2024

A multimodal LLM for the non-invasive decoding of spoken text from brain recordings

Youssef Hmamouche, Ismail Chihab, Lahoucine Kdouri, Amal El Fallah Seghrouchni

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments 15 pages, 4 figures

dnaGrinder: a lightweight and high-capacity genomic foundation model

Qihang Zhao, Chi Zhang, Weixiong Zhang

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Source-Aware Training Enables Knowledge Attribution in Language Models

Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, Hao Peng

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；pretraining(abstract)

Comments COLM '24

IGOT: Information Gain Optimized Tokenizer on Domain Adaptive Pretraining

Dawei Feng, Yihai Zhang, Zhixuan Xu

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web

Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmermann, Yuxuan Liang

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by The Web Conference 2024

Learning to Deliver: a Foundation Model for the Montreal Capacitated Vehicle Routing Problem

Samuel J. K. Chin, Matthias Winkenbach, Akash Srivastava

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Biao Zhang, Zhongtao Liu, Colin Cherry, Orhan Firat

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments ICLR24

She had Cobalt Blue Eyes: Prompt Testing to Create Aligned and Sustainable Language Models

Veronica Chatrath, Oluwanifemi Bamgbose, Shaina Raza

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；prompting(abstract)

Comments Accepted as Oral at the AAAI 2nd Workshop on Sustainable AI

Proving Test Set Contamination in Black Box Language Models

Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, Tatsunori B. Hashimoto

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；prompting(abstract)

Skywork: A More Open Bilingual Foundation Model

Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Haihua Yang, Biye Li, Cheng Cheng, Weiwei Lü, Rui Hu, Chenxia Li, Liu Yang, Xilin Luo, Xuejie Wu, Lunan Liu, Wenjun Cheng, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Lei Lin, Xiaokun Wang, Yutuan Ma, Chuanhai Dong, Yanqi Sun, Yifu Chen, Yongyi Peng, Xiaojuan Liang, Shuicheng Yan, Han Fang, Yahui Zhou

专题命中 预训练与数据 ：foundation model(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization

Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, Junxian He

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；instruction tuning(abstract)

The Wisdom of Hindsight Makes Language Models Better Instruction Followers

Tianjun Zhang, Fangchen Liu, Justin Wong, Pieter Abbeel, Joseph E. Gonzalez

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；RLHF(abstract)

General Intelligence Requires Reward-based Pretraining

Seungwook Han, Jyothish Pari, Samuel J. Gershman, Pulkit Agrawal

机构 * Improbable AI Lab, MIT（Improbable AI Lab，麻省理工学院） ； Department of Psychology（心理学系） ； Center for Brain Science, Harvard University（哈佛大学脑科学中心） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments https://improbableai.notion.site/General-Intelligence-Requires-Reward-Based-Pretraining-2023b66e4cf580d3ab44c7860b75d25f?pvs=73

SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challenge

Yuxiang Mei, Yuang Zheng, Dongxing Xu, Yanhua Long

机构 * Shanghai Normal University（上海师范大学） ； Unisound AI Technology Co., Ltd.（Unisound人工智能技术有限公司） 纠错

专题命中 预训练与数据 ：SLM(title,abstract)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by Interspeech 2025 MLC-SLM workshop

XRFix: Exploring Performance Bug Repair of Extended Reality Applications with Large Language Models

XRFix：利用大语言模型探索扩展现实应用的性能缺陷修复

Jingwen Wu, Hanyang Guo, Hong-Ning Dai, Xiapu Luo

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 该研究针对XR应用性能缺陷修复的技术挑战，提出基于大语言模型的XRFix框架，构建缺陷数据集与定制检测工具，经实验验证其修复性能优于SOTA APR方法。

Journal ref Proceedings of the 48th International Conference on Software Engineering (ICSE 2026)

BrainNorm: A Foundation Model that knows Normal via Semantic Atlas Pretraining

BrainNorm：一种通过语义图谱预训练了解正常状态的基础模型

Madhumitha Venkatesh, Shanawaj S Madarkar, Konda Reddy Mopuri

机构 * Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)

AI总结 该研究提出BrainNorm基础模型，经约6.6万例T1w sMRI数据训练，通过语义图谱预训练学习规范表征，在多类下游任务上泛化性能优异，其识别的神经退行性偏差与临床病理吻合。

Know3D: Prompting 3D Generation with Knowledge from Vision-Language Models

Know3D: 通过视觉-语言模型的知识提示3D生成

Wenyue Chen, Wenjue Chen, Peng Li, Qinghe Wang, Xu Jia, Heliang Zheng, Rongfei Jia, Yuan Liu, Ronggang Wang

机构 * Peking University（北京大学） ； Math Magic（数学魔术） ； The Hong Kong University of Science and Technology（香港科学与技术大学） ； Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology（广东省超高清沉浸媒体技术重点实验室） ； Dalian University of Technology（大连理工大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；prompting(title)；large language model(abstract)

AI总结 本文提出Know3D框架，利用多模态大语言模型的知识提升3D生成的可控性，通过潜在状态注入实现语言可控的后视图生成，改进3D生成的几何细节与一致性。

Comments ECCV2026 page: https://xishuxishu.github.io/Know3D.github.io/

CoGate: Confidence-Gated Co-Decoding for Secure Code Generation

CoGate：用于安全代码生成的置信门控协同解码

Minghao Hu, Lannan Luo, Allen Roush, Phillip Howard

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结 针对现有协同解码未考虑专家模型置信度的问题，提出CoGate方法，在多LLM后端和代码生成基准上优于CoSec+，在CWEval的Func-Sec@10指标最高提升12.6%。

Auditing pretraining contamination in single-cell foundation model benchmarks

单细胞基础模型基准中的预训练污染审计

Sarwan Ali

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)

AI总结 研究单细胞基础模型基准中的预训练污染问题，提出scContam审计框架，结合指纹信号与成员推理攻击。应用于多个基准和模型，发现部分基准有污染，且MIA-scFM的AUROC与模型容量数据比有关，证明预训练审计可行且应伴随基准报告。

The CRAFT principles for the responsible use of large language models in policymaking

政策制定中负责任使用大语言模型的CRAFT原则

Willem Fourie, Gray Manicom, Tanya de Villiers-Botha

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 研究政策制定中如何利用大语言模型，提出CRAFT原则，即控制、严谨、问责、公平和透明，以在管理风险的同时充分利用大语言模型改善政策制定过程。

Alternative routes to universal diversity scaling in component systems: from proteomes to large language models

组件系统中通用多样性标度的替代路径：从蛋白质组到大语言模型

Andrea Mazzolini, Leonardo Agasso, Filippo Valle, Michele Caselle, Marco Cosentino Lagomarsino, Matteo Osella

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)

AI总结 研究复杂组件系统中多样性标度定律的涌现条件，发现创新驱动增长模型与潜在异质性模型均能产生相同的宏观定律，但无法唯一确定生成机制。

Understanding the (In)Security of Vibe-Coded Applications

理解“氛围编码”应用的安全性（与非安全性）

Junquan Deng, Zhiyu Fan, Ruijie Meng

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

AI总结 本文系统研究了通过“氛围编码”范式开发的应用的安全性，发现其存在占位符逻辑、未过滤输入和秘密泄露等重复性漏洞模式，根源在于AI代理的系统性局限，且即使改进LLM能力也无法消除根本风险。

Formal-Method-Guided Vibe Coding: Closing the Verification Loop on AI-Generated Safety-Critical Software Through Model-Driven Engineering

形式化方法引导的Vibe编码：通过模型驱动工程关闭AI生成安全关键软件的验证循环

Ran Wei, Le Zhu, Haochi Wang, Jim Woodcock, Fang Yan, Simon Foster, Xiangyang Ji

专题命中 预训练与数据 ：LLM(summary_cn,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结 提出Forge管道，结合Vibe编码与模型驱动工程，通过多种形式化验证（Dafny、CSP、Isabelle）迭代修正LLM生成的Java代码，生成符合安全标准的验证证据。

Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining

Brain-DiT：一种基于元数据条件预训练的通用多状态fMRI基础模型

Junfeng Xia, Wenhao Ye, Xuanye Pan, Xinke Shen, Mo Wang, Quanying Liu

机构 * Department of Biomedical Engineering, Southern University of Science and Technology, China（南方科技大学生物医学工程系，中国） ； School of Biomedical Engineering, Shenzhen University, China（深圳大学生物医学工程学院，中国） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)

AI总结 提出Brain-DiT，一种基于扩散Transformer和元数据条件预训练的通用多状态fMRI基础模型，在24个数据集上预训练，通过生成式预训练学习多尺度表示，在7个下游任务中优于重建和对齐方法。