语言大模型 / LLM

MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series

Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Esther Cheng, Jie Liu, Qunshu Lin, Raven Yuan, Tuney Zheng, Wei Pang, Xinrun Du, Yiming Liang, Yinghao Ma, Yizhi Li, Ziyang Ma, Bill Lin, Emmanouil Benetos, Huan Yang, Junting Zhou, Kaijing Ma, Minghao Liu, Morry Niu, Noah Wang, Quehry Que, Ruibo Liu, Sine Liu, Shawn Guo, Soren Gao, Wangchunshu Zhou, Xinyue Zhang, Yizhi Zhou, Yubo Wang, Yuelin Bai, Yuhan Zhang, Yuxiang Zhang, Zenith Wang, Zhenzhu Yang, Zijian Zhao, Jiajun Zhang, Wanli Ouyang, Wenhao Huang, Wenhu Chen

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments https://map-neo.github.io/

Large Language Models Can Infer Psychological Dispositions of Social Media Users

Heinrich Peters, Sandra Matz

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Libra: Building Decoupled Vision System on Large Language Models

Yifan Xu, Xiaoshan Yang, Yaguang Song, Changsheng Xu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Comments ICML2024

Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair

Yusuke Sakai, Mana Makinae, Hidetaka Kamigaito, Taro Watanabe

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages, 9 figures

LLaFS: When Large Language Models Meet Few-Shot Segmentation

Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Accepted to CVPR2024

Evaluating Large Language Models for Health-Related Text Classification Tasks with Public Social Media Data

Yuting Guo, Anthony Ovadje, Mohammed Ali Al-Garadi, Abeed Sarker

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

A Moral Imperative: The Need for Continual Superalignment of Large Language Models

Gokul Puthumanaillam, Manav Vora, Pranay Thangeda, Melkior Ornik

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Yang Jin, Kun Xu, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Quzhe Huang, Bin Chen, Chenyi Lei, An Liu, Chengru Song, Xiaoqiang Lei, Di Zhang, Wenwu Ou, Kun Gai, Yadong Mu

专题命中 预训练与数据 ：LLM(title,abstract)；pretraining(title)；large language model(abstract)；language model(abstract)

Comments ICLR 2024

Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models

Kang Gu, Md Rafi Ur Rashid, Najrin Sultana, Shagufta Mehnaz

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

AXOLOTL: Fairness through Assisted Self-Debiasing of Large Language Model Outputs

Sana Ebrahimi, Kaiwen Chen, Abolfazl Asudeh, Gautam Das, Nick Koudas

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes

Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Tong Yu, Hanieh Deilamsalehy, Ruiyi Zhang, Sungchul Kim, Franck Dernoncourt

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Towards Trustable Language Models: Investigating Information Quality of Large Language Models

Rick Rejeleene, Xiaowei Xu, John Talburt

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 31 pages

Deep de Finetti: Recovering Topic Distributions from Large Language Models

Liyi Zhang, R. Thomas McCoy, Theodore R. Sumers, Jian-Qiao Zhu, Thomas L. Griffiths

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 13 pages, 4 figures

Data-Centric Financial Large Language Models

Zhixuan Chu, Huaiyu Guo, Xinyuan Zhou, Yijia Wang, Fei Yu, Hong Chen, Wanqing Xu, Xin Lu, Qing Cui, Longfei Li, Jun Zhou, Sheng Li

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Irreducible Curriculum for Language Model Pretraining

Simin Fan, Martin Jaggi

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias

Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, Chao Zhang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NeurIPS 2023 (Datasets and Benchmarks Track)

Journal ref NeurIPS 2023

Enhancing Pipeline-Based Conversational Agents with Large Language Models

Mina Foosherian, Hendrik Purwins, Purna Rathnayake, Touhidul Alam, Rui Teimao, Klaus-Dieter Thoben

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Inductive-bias Learning: Generating Code Models with Large Language Model

Toma Tanaka, Naofumi Emoto, Tsukasa Yumibayashi

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 17 pages, 8 figures

D4: Improving LLM Pretraining via Document De-Duplication and Diversification

Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari S. Morcos

专题命中 预训练与数据 ：LLM(title,abstract)；pretraining(title)；large language model(abstract)；language model(abstract)

Procedural Pretraining: Warming Up Language Models with Abstract Data

程序化预训练：用抽象数据预热语言模型

Liangze Jiang, Zachary Shinnick, Anton van den Hengel, Hemanth Saratchandran, Damien Teney

机构 * EPFL（苏黎世联邦理工学院） ； Idiap Research Institute（伊迪普研究机构） ； AIML, Adelaide University（人工智能实验室，阿德莱德大学） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(title,abstract)；LLM(abstract_cn)；分类 cs.CL、cs.LG

AI总结 提出程序化预训练方法，通过在抽象结构化数据（如形式语言生成的程序数据）上预训练语言模型，显著提升其推理能力并加速后续语义知识学习，实验表明仅需0.1-0.3%的程序数据即可超越标准预训练。

Comments ICML 2026. Project page: https://zlshinnick.github.io/procedural-pretraining-page/

Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models

Badrinath Ramakrishnan, Akshaya Balaji

机构 * Badrinath Ramakrishnan Akshaya Balaji（独立研究者） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI

Comments 14 pages, 2 figures. Code and experimental framework available at https://github.com/akshayaaa10/llm-privacy-research

How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance

Desiree Heim, Lars-Peter Meyer, Markus Schröder, Johannes Frey, Andreas Dengel

专题命中 预训练与数据 ：language model(title,abstract)；large language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Peer reviewed and to appear in the ESWC 2025 Workshops and Tutorials Joint Proceedings (Workshop on Evaluation of Language Models in Knowledge Engineering [ELMKE])

EDA Corpus: A Large Language Model Dataset for Enhanced Interaction with OpenROAD

Bing-Yue Wu, Utsav Sharma, Sai Rahul Dhanvi Kankipati, Ajay Yadav, Bintu Kappil George, Sai Ritish Guntupalli, Austin Rovinski, Vidya A. Chhabria

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI

Comments Under review at Workshop on LLM-Aided Design (LAD'24)

Single-Round Vector RAG vs an LLM-Compiled Wiki: A Preregistered Comparison on a Small Multi-Domain Research Corpus

向量RAG与LLM编写的维基：在小型多领域研究上的预注册比较

Theodore O. Cochran

机构 * AI for Altruism (A4A)（AI为利他主义（A4A）） 纠错

专题命中 预训练与数据 ：LLM(title,title_cn)；分类 cs.CL

AI总结 本文通过预注册比较，研究了向量RAG系统与LLM编写的维基在小型多领域研究中的表现，发现两者在跨论文连接、答案组织和成本等方面各有优劣，没有单一系统在所有指标上最优。

Comments v2: two-judge reanalysis of the decomposition-RAG ablation (groundedness advantage +1.15 to +0.15); H3a adjudicated; one registered-plan deviation disclosed; artifact deposit at osf.io/j37b8; title corrected

TEAMMix: Taxonomy Enrichment Augmentation and Minority-augmented Mixing Strategy for LLM-enhanced Weak-Supervised Hierarchical Text Classification

TEAMMix：用于大语言模型增强弱监督分层文本分类的类别体系丰富增强与少数类增强混合策略

Jian Zhang, Zhuohao Yang, Songlin Lei, Bangli Liu, Ziwei Wang, Xufeng Weng, Gehan Amaratunga, Yu Lin, Hongwei Wang

机构 * School of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC研究院） ； Shaoxing K3i Technology Co. Ltd（绍兴K3i科技有限公司） ； State Key Laboratory of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室） 纠错

专题命中 预训练与数据 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL

AI总结 针对LLM应用于分层文本分类的局限，本文提出TEAMMix框架，通过丰富标签层次、生成伪样本并优化其质量，提升了细粒度及不平衡数据集上的分类性能。

Comments Accepted by IEEE CSCWD 2026

Scalable Frequency- and Length-Aware Subdocument Deduplication for Large Language Model Pretraining

面向大语言模型预训练的可扩展、感知频率与长度的子文档去重

Hai Wang, Chenhao Wang, Qifeng Cai, Yixiu Liu, Miao Peng, Nuo Chen, Yuanlin Tu, Chengcheng Xu, Feng Zhang

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(title)；language model(title)；分类 cs.CL

AI总结 针对大语言模型预训练的子文档去重难题，提出解耦重复检测与副本保留的可扩展框架，在基准数据集上实现最优模型性能，凸显显式副本保留控制的价值。

Deep Research Pretraining via Predictive Navigation

基于预测导航的深度研究预训练

Jiang Zhou, Zhiyuan Fan, Xing Wu, Tinghao Yu, Feng Zhang, Lilin Wang

机构 * Tencent（腾讯） ； Hunyuan Team（混元团队） 纠错

专题命中 预训练与数据 ：pretraining(title,summary_cn)；SFT(abstract,abstract_cn)；分类 cs.CL

AI总结 提出Deep Research Pretraining（DRP）离线框架，在学术引用图和维基百科超链接上预训练Qwen3-14B-Base模型，可提升智能体在多个基准任务上的表现，是轨迹智能体训练的补充方法。

Comments working in progress; correspondence to {ucaswu,maxwellyu}@tencent.com or wuxing@iie.ac.cn

Beyond Captions: Context-Grounded Reconstruction for Biomedical Multimodal Continued Pretraining

PMC-InterCPT：重新思考用于多模态持续预训练的医学交错数据

Guanghao Zhu, Zeyu Liu, Zhitian Hou, Pengkai Wang, Zhijie Sang, Yang Yu, Minheng Ni, Wenjun Wang, Yanggan Gu, Shuo Cai, Congkai Xie, Jianmin Wu, Hongxia Yang

机构 * The Hong Kong Polytechnic University（香港理工大学） ； Sun Yat-sen University（中山大学） ； InfiX.ai ； PolyU-Daya Bay Technology and Innovation Research Institute（PolyU-大亚湾技术与创新研究院） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(summary_cn,abstract_cn)；SFT(abstract,abstract_cn)；分类 cs.CL

AI总结 针对医学多模态持续预训练中图像-文本对数据存在的上下文缺失、结构噪声等问题，提出PMC-InterCPT交错语料库，通过恢复缺失标题、清洗文本、重建交错样本及LLM监督过滤，结合模态感知重采样，有效提升医学与通用多模态性能。

EvoCause: LLM-Guided Evolution of Causal Graphs for Root Cause Analysis

EvoCause：基于大语言模型引导的因果图进化的根本原因分析

Lei Zan, Keli Zhang, Shifeng Xie, Jiale Zheng, Zehao Xiao, Zhiwei Dong, Ke Zhang, Ruichu Cai, Malik Tiomoko, Lujia Pan

专题命中 预训练与数据 ：LLM(title,summary_cn)；large language model(abstract)；language model(abstract)；分类 cs.LG

AI总结 EvoCause 用 LLM 优化因果图以提升电信等系统的根本原因分析性能，发布了 TeleRCA 基准，在合成数据和真实网络数据上均优于基线方法。

Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations

选择还是投影？评估用于LLM训练数据解释的低维向量

Lukas Hinterleitner, Loris Schoenegger, Benjamin Roth

机构 * Faculty of Computer Science, University of Vienna（维也纳大学计算机科学系） ； UniVie Doctoral School Computer Science, University of Vienna（维也纳大学计算机科学博士学院） ； Faculty of Philological and Cultural Studies, University of Vienna（维也纳大学文学与文化研究系） 纠错

专题命中 预训练与数据 ：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL

AI总结 本文通过对比选择和投影方法，发现有针对性的组件选择在LLM训练数据解释中更有效且计算更高效。

Comments KONVENS 2026. 9 pages