语言大模型 / LLM

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

BLISS: 一种用于语言模型预训练数据选择的轻量级双层影响评分方法

Jie Hao, Rui Yu, Wei Zhang, Huixia Wang, Jie Xu, Mingrui Liu

机构 * Department of Computer Science, George Mason University, USA（乔治·马歇尔大学计算机科学系） ； IBM T.J. Watson Research Center, USA（IBM T.J. Watson研究部） ； Department of Statistics, Rice University（里士大学统计系） ； Department of System Engineering & Operations Research, George Mason University, USA（乔治·马歇尔大学系统工程与运营管理系） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；LLM(abstract,abstract_cn)；large language model(abstract)

AI总结 提出一种无需外部预训练模型的轻量级数据选择方法BLISS，通过双层优化和代理模型估计训练样本的长期影响，实现高效数据筛选，在C4数据集上预训练多种规模模型，显著加速收敛并提升下游任务性能。

Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition

Dango：一个严格仅L1的大型语言模型，用于研究第二语言习得

Shiho Matta, Yin Jou Huang, Fei Cheng, Takashi Kodama, Hirokazu Kiyomaru, Yugo Murawaki

机构 * Kyoto University（京都大学） ； NII-LLMC（日本国立信息与通信技术研究所-语言模型中心） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,abstract_cn)；pretraining(abstract)

AI总结 提出1.8B参数的Dango模型，通过过滤L2污染和微调L2学习课程，模拟人类L2产出模式，优于未过滤和多语言基线。

Comments 8 pages main text, 20 pages total including references and appendices

AWM: Accurate Weight-Matrix Fingerprint for Large Language Models

AWM: 用于大型语言模型的准确权重矩阵指纹

Boyi Zeng, Lin Chen, Ziwei He, Xinbing Wang, Zhouhan Lin

机构 * LUMIA Lab（LUMIA实验室） ； School of Artificial Intelligence（人工智能学院） ； Shanghai Jiao Tong University（上海交通大学） ； Shanghai Innovation Institute（上海创新研究院） ； Fudan University（复旦大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

AI总结 AWM通过基于权重矩阵的无训练指纹方法，利用线性分配问题和无偏中心核对齐相似性，实现对大型语言模型训练来源的可靠识别。

Comments ICLR 2026

It's All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models

信心才是关键：一种用于多语言历史实体链接的无监督方法，使用大语言模型

Cristian Santini, Marieke Van Erp, Mehwish Alam

机构 * Department of Humanities, University of Macerata（马切拉塔大学人文学科系） ； KNAW Humanities Cluster, DHLab（荷兰人文集群、DHLab） ； INFRES Department, Télécom Paris（巴黎电信学院INFRES部门） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；small language model(abstract)

AI总结 本文提出MHEL-LLaMo，一种结合小型语言模型和大语言模型的无监督方法，用于多语言历史实体链接，通过置信度评分降低计算成本并提升准确性。

AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization

Fabrycio Leite Nakano Almada, Kauan Divino Pouso Mariano, Maykon Adriell Dutra, Victor Emanuel da Silva Monteiro, Juliana Resplande Sant'Anna Gomes, Arlindo Rodrigues Galvão Filho, Anderson da Silva Soares

机构 * Institute of Informatics, Federal University of Goiás, Brazil（信息学院，戈亚斯联邦大学，巴西） ； Advanced Knowledge Center in Immersive Technology (AKCIT), Federal University of Goiás, Brazil（沉浸式技术高级知识中心（AKCIT），戈亚斯联邦大学，巴西） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；prompting(title,abstract)；large language model(abstract)；language model(abstract)

Comments 15 pages, 2 figures

Llamarine: Open-source Maritime Industry-specific Large Language Model

William Nguyen, An Phan, Konobu Kimura, Hitoshi Maeno, Mika Tanaka, Quynh Le, William Poucher, Christopher Nguyen

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Comments Work in progress

Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language Models

Ilias Stogiannidis, Stavros Vassos, Prodromos Malakasiotis, Ion Androutsopoulos

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Short paper (5 pages), accepted at Findings of EMNLP 2023

Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring

谁来评判评判者？基于指标的治理：面向持续LLM合规监控的运行时框架

Jehanne Dussert

机构 * Independent Researcher（独立研究者） 纠错

专题命中 预训练与数据 ：LLM(title,title_cn)；language model(abstract)；small language model(abstract)；SLM(abstract_cn)

AI总结 针对AI合规作为审计时二元判定而非生产系统持续可测量属性的问题，提出基于指标的治理原则，并开发开源框架govllm，通过运行时可观测性信号实现持续合规监控，验证了多模型陪审团设计在监管评估中的有效性。

Comments 41 pages, 8 figures, preprint

Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity

对比LLM和微调模型在不同提示复杂度下的NVDRS情境提取性能

Geoffrey Martin, Xuan Zhong Feng, Yifan Peng

机构 * Department of Population Health Sciences, Weill Cornell Medicine（人群健康科学系，威尔·康奈尔医学学院） ； Systems Engineering, Cornell University（系统工程，康奈尔大学） 纠错

专题命中 预训练与数据 ：LLM(title,title_cn)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结 本文研究了在不同提示复杂度下，LLM与微调模型在NVDRS情境提取任务中的表现差异，提出了一种复杂度评分算法，并展示了一个混合方法，通过不同情境选择提示策略，发现LLM在低 prevalence 情境中表现更优，且框架能跨不同前沿LLM通用。

Comments Accepted at IEEE ICHI 2026

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining

Ping Guo, Yubing Ren, Binbin Liu, Fengze Liu, Haobin Lin, Yifan Zhang, Bingni Zhang, Taifeng Wang, Yin Zheng

机构 * ByteDance（字节跳动） ； Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(title)；分类 cs.CL、cs.AI

Retrieval Capabilities of Large Language Models Scale with Pretraining FLOPs

Jacob Portes, Connor Jennings, Erica Ji Yuen, Sasha Doubov, Michael Carbin

机构 * Databricks Mosaic Research（Databricks Mosaic研究） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(title)；language model(title)；LLM(abstract)

Comments 15 pages, 4 figures

CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models

Guang Liu, Liangdong Wang, Jijie Li, Yang Yu, Yao Xu, Jiabei Chen, Yu Bai, Feng Liao, Yonghua Lin

机构 * Data Research Team（数据研究团队） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(title)；language model(title)；LLM(abstract)

KnowledgePrompts: Exploring the Abilities of Large Language Models to Solve Proportional Analogies via Knowledge-Enhanced Prompting

Thilini Wijesiriwardene, Ruwan Wickramarachchi, Sreeram Vennam, Vinija Jain, Aman Chadha, Amitava Das, Ponnurangam Kumaraguru, Amit Sheth

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(title)；分类 cs.CL、cs.AI

Comments Accepted at COLING 2025

All Languages Matter: On the Multilingual Safety of Large Language Models

Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao, Michael R. Lyu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；prompting(abstract)

Comments Accepted by ACL 2024 Findings. The first multilingual safety benchmark for large language models

LLMR: Real-time Prompting of Interactive Worlds using Large Language Models

Fernanda De La Torre, Cathy Mengying Fang, Han Huang, Andrzej Banburski-Fahey, Judith Amores Fernandez, Jaron Lanier

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；prompting(title)；分类 cs.CL、cs.AI

Comments 46 pages, 18 figures; Matching version accepted at CHI 2024

Panda LLM: Training Data and Evaluation for Open-Sourced Chinese Instruction-Following Large Language Models

Fangkai Jiao, Bosheng Ding, Tianze Luo, Zhanfeng Mo

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(title)；分类 cs.CL、cs.AI

AMALIA Technical Report: A Fully Open Source Large Language Model for European Portuguese

AMALIA技术报告：一个完全开源的大型语言模型用于葡萄牙语（欧洲葡萄牙语）

Afonso Simplício, Gonçalo Vinagre, Miguel Moura Ramos, Diogo Tavares, Rafael Ferreira, Giuseppe Attanasio, Duarte M. Alves, Inês Calvo, Inês Vieira, Rui Guerra, James Furtado, Beatriz Canaverde, Iago Paulo, Vasco Ramos, Diogo Glória-Silva, Miguel Faria, Marcos Treviso, Daniel Gomes, Pedro Gomes, David Semedo, André Martins, João Magalhães

机构 * NOVA School of Science and Technology（新大学科学与技术学院） ； NOVA LINCS（新大学LINCS实验室） ； Instituto de Telecomunicações（电信研究所） ； Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院） ； Fundação para a Ciência e Tecnologia（科学技术基金会） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

AI总结 AMALIA是一个完全开源的大型语言模型，专注于欧洲葡萄牙语，通过增加高质量数据提升模型性能，并发布新基准测试以评估葡萄牙语生成和语言能力。

Comments PROPOR 2026 - The 17th International Conference on Computational Processing of Portuguese

Large Language Models for Travel Behavior Prediction

基于大语言模型的出行行为预测

Baichuan Mo, Hanyong Xu, Ruoyun Ma, Jung-Hoon Cho, Dingyi Zhuang, Xiaotong Guo, Jinhua Zhao

机构 * Department of Civil Engineering, Tsinghua University, Beijing, China（清华大学土木工程系） ； Department of Civil and Environmental Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139（麻省理工学院土木与环境工程系） ； Department of Urban Studies and Planning, Massachusetts Institute of Technology, Cambridge, MA 20139（麻省理工学院城市研究与规划系） ； Department of Management Science and Engineering, Stanford University, Stanford, CA 94305（斯坦福大学管理科学与工程系） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本文提出利用大语言模型进行出行行为预测，通过零样本提示和文本嵌入两种方法，实现了与传统模型相当的预测性能。

Can Safety Emerge from Weak Supervision? A Systematic Analysis of Small Language Models

安全能否从弱监督中涌现？小型语言模型的系统分析

Punyajoy Saha, Sudipta Halder, Debjyoti Mondal, Subhadarshi Panda

机构 * Samsung Research Institute（三星研究院） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；small language model(title,abstract)；large language model(abstract)；preference optimization(abstract)

AI总结 Self-MOA通过弱监督实现小型语言模型的安全对齐，显著提升安全性的同时保持有用性，减少对人工标注数据的依赖。

Comments 19 pages, 10 tables, 7 figures, under Review

STELLA: Guiding Large Language Models for Time Series Forecasting with Semantic Abstractions

STELLA: 通过语义抽象引导大型语言模型进行时间序列预测

Junjie Fan, Hongye Zhao, Linduo Wei, Jiayu Rao, Guijia Li, Jiaxin Yuan, Wenqi Xu, Yong Qi

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 STELLA通过动态语义抽象机制，提升大型语言模型在时间序列预测中的表现，实现更精准的长期和短期预测。

Comments This work has been submitted to the IEEE for possible publication

Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimization

Wataru Kawakami, Keita Suzuki, Junichiro Iwasawa

机构 * Preferred Networks Inc.（Preferred Networks公司） ； Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；preference optimization(title,abstract)；large language model(abstract)；language model(abstract)

Fourier Head: Helping Large Language Models Learn Complex Probability Distributions

Nate Gillman, Daksh Aggarwal, Michael Freeman, Saurabh Singh, Chen Sun

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Comments Camera ready version (ICLR 2025). Code at https://nategillman.com/fourier-head

LoLCATs: On Low-Rank Linearizing of Large Language Models

Michael Zhang, Simran Arora, Rahul Chalamala, Alan Wu, Benjamin Spector, Aaryan Singhal, Krithik Ramesh, Christopher Ré

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments 58 pages, 25 figures, 26 tables, ICLR 2025

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data

Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, William Yang Wang

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；LLM(abstract)；large language model(abstract)

Comments Accepted to ICLR 2025

Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective

Zeyu Gan, Yong Liu

专题命中 预训练与数据 ：LLM(title,abstract)；post-training(title,abstract)；large language model(abstract)；language model(abstract)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining

Ce Ge, Zhijian Ma, Daoyuan Chen, Yaliang Li, Bolin Ding

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；LLM(abstract)；large language model(abstract)

Comments Clarify details

Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability

Zhuoyan Xu, Zhenmei Shi, Yingyu Liang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models

Yixin Ou, Ningyu Zhang, Honghao Gui, Ziwen Xu, Shuofei Qiao, Yida Xue, Runnan Fang, Kangwei Liu, Lei Li, Zhen Bi, Guozhou Zheng, Huajun Chen

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；prompting(abstract)

Comments ACL 2024 System Demonstrations; Project website: https://zjunlp.github.io/project/EasyInstruct Code: https://github.com/zjunlp/EasyInstruct Video: https://youtu.be/rfQOWYfziFo Demo: https://huggingface.co/spaces/zjunlp/EasyInstruct

In-context Pretraining: Language Modeling Beyond Document Boundaries

Weijia Shi, Sewon Min, Maria Lomeli, Chunting Zhou, Margaret Li, Gergely Szilvasy, Rich James, Xi Victoria Lin, Noah A. Smith, Luke Zettlemoyer, Scott Yih, Mike Lewis

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；prompting(abstract)

GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models

Ruotong Liao, Xu Jia, Yangzhe Li, Yunpu Ma, Volker Tresp

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；instruction tuning(abstract)

Comments 14 pages, Findings of NAACL 2024, Spotlight on TGL@NeurIPS2023