语言大模型 / LLM

LabelFusion: Fusing Large Language Models with Transformer Encoders for Robust Financial News Classification

LabelFusion：融合大型语言模型与Transformer编码器以实现稳健的金融新闻分类

Michael Schlee, Christoph Weisser, Timo Kivimäki, Melchizedek Mashiku, Benjamin Saefken

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本文提出LabelFusion架构，结合提示工程的LLM与微调的RoBERTa编码器，通过轻量MLP投票层提升金融新闻分类性能，在低数据环境下LLM单独使用表现更优。

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection

通过基于模型的数据选择增强多语言大语言模型预训练

Bettina Messmer, Vinko Sabolčec, Martin Jaggi

机构 * EPFL（苏黎世联邦理工学院） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；pretraining(title,abstract)；large language model(abstract)；language model(abstract)

AI总结 本文提出了一种基于模型的数据选择框架，通过提高多语言大语言模型预训练的效果，实现了在较少训练数据下达到基线分数并提升其他基准测试的表现。

Comments NeurIPS 2025 Track on Datasets and Benchmarks

Generalizing Scaling Laws for Dense and Sparse Large Language Models

为密集和稀疏大语言模型推广缩放定律

Md Arafat Hossain, Xingfu Wu, Valerie Taylor, Ali Jannesari

机构 * Department of Computer Science（计算机科学系） ； Iowa State University（爱荷华州立大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

AI总结 本文提出了一种通用缩放定律，适用于密集和稀疏大语言模型，通过比较实验验证了其在大规模模型中的有效性。

Comments 8 pages, 8 figures

Can Large Language Models Capture Video Game Engagement?

大语言模型能否捕捉视频游戏参与度？

David Melhart, Matthew Barthet, Georgios N. Yannakakis

机构 * Institute of Digital Games, University of Malta Msida, Malta（马耳他大学数字游戏研究所） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

AI总结 本文研究了大语言模型在多模态输入下预测视频游戏参与度的能力，发现尽管LLMs在多个领域表现优异，但其在连续情绪标注上仍无法超越人类注释。

Comments This work has been submitted to the IEEE for publication

The Data Efficiency Frontier of Financial Foundation Models: Scaling Laws from Continued Pretraining

金融基础模型的数据效率前沿：从持续预训练中获得的扩展定律

Jesse Ponnock

机构 * Johns Hopkins University（约翰霍普金斯大学） 纠错

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；language model(abstract)

AI总结 本文研究了金融基础模型在持续预训练中的数据效率，发现通过较小的词预算可实现有效的领域适应，且大模型规模仍具可行性。

Comments 8 pages, 4 figures, 1 table

LLMLagBench: Identifying Temporal Training Boundaries in Large Language Models

Piotr Pęzik, Konrad Kaczyński, Maria Szymańska, Filip Żarnecki, Zuzanna Deckert, Jakub Kwiatkowski, Wojciech Janowski

机构 * University of Lodz（洛兹大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

On the Military Applications of Large Language Models

Satu Johansson, Taneli Riihonen

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；pretraining(abstract)

Comments Published in the meeting proceedings of the 2025 International Conference on Military Communications and Information Systems (ICMCIS) by the NATO Science and Technology Organization (STO)

Hallucinate or Memorize? The Two Sides of Probabilistic Learning in Large Language Models

Junichiro Niimi

机构 * Meijo University（名古屋大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

PLLuM: A Family of Polish Large Language Models

Jan Kocoń, Maciej Piasecki, Arkadiusz Janz, Teddy Ferdinan, Łukasz Radliński, Bartłomiej Koptyra, Marcin Oleksy, Stanisław Woźniak, Paweł Walkowiak, Konrad Wojtasik, Julia Moska, Tomasz Naskręt, Bartosz Walkowiak, Mateusz Gniewkowski, Kamil Szyc, Dawid Motyka, Dawid Banach, Jonatan Dalasiński, Ewa Rudnicka, Bartłomiej Alberski, Tomasz Walkowiak, Aleksander Szczęsny, Maciej Markiewicz, Tomasz Bernaś, Hubert Mazur, Kamil Żyta, Mateusz Tykierko, Grzegorz Chodak, Tomasz Kajdanowicz, Przemysław Kazienko, Agnieszka Karlińska, Karolina Seweryn, Anna Kołos, Maciej Chrabąszcz, Katarzyna Lorenc, Aleksandra Krasnodębska, Artur Wilczek, Katarzyna Dziewulska, Paula Betscher, Zofia Cieślińska, Katarzyna Kowol, Daria Mikoś, Maciej Trzciński, Dawid Krutul, Marek Kozłowski, Sławomir Dadas, Rafał Poświata, Michał Perełkiewicz, Małgorzata Grębowiec, Maciej Kazuła, Marcin Białas, Roman Roszko, Danuta Roszko, Jurgita Vaičenonienė, Andrius Utka, Paweł Levchuk, Paweł Kowalski, Irena Prawdzic-Jankowska, Maciej Ogrodniczuk, Monika Borys, Anna Bulińska, Wiktoria Gumienna, Witold Kieraś, Dorota Komosińska, Katarzyna Krasnowska-Kieraś, Łukasz Kobyliński, Martyna Lewandowska, Marek Łaziński, Mikołaj Łątkowski, Dawid Mastalerz, Beata Milewicz, Agnieszka Anna Mykowiecka, Angelika Peljak-Łapińska, Sandra Penno, Zuzanna Przybysz, Michał Rudolf, Piotr Rybak, Karolina Saputa, Aleksandra Tomaszewska, Aleksander Wawer, Marcin Woliński, Joanna Wołoszyn, Alina Wróblewska, Bartosz Żuk, Filip Żarnecki, Konrad Kaczyński, Anna Cichosz, Zuzanna Deckert, Monika Garnys, Izabela Grabarczyk, Wojciech Janowski, Sylwia Karasińska, Aleksandra Kujawiak, Piotr Misztela, Maria Szymańska, Karolina Walkusz, Igor Siek, Jakub Kwiatkowski, Piotr Pęzik

机构 * Department of Artificial Intelligence, Wrocław University of Science and Technology（沃斯克拉大学人工智能系） ； NASK National Research Institute（国家研究 institute） ； National Information Processing Institute（国家信息处理研究所） ； Institute of Slavic Studies, Polish Academy of Sciences（波兰科学院斯拉夫研究学院） ； Institute of Computer Science, Polish Academy of Sciences（波兰科学院计算机科学研究所） ； University of Łódź（卢布林大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；preference optimization(abstract)

Comments 83 pages, 19 figures

Electronic Circuit Principles of Large Language Models

Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiaqi Wang, Mengkang Hu, Zhi Chen, Wanxiang Che, Ting Liu

机构 * Harbin Institute of Technology（哈尔滨工业大学） ； Central South University（中南大学） ； The Chinese University of Hong Kong（香港中文大学） ； The University of Hong Kong（香港大学） ； ByteDance Seed (China)（字节跳动种子（中国）） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Manuscript

Llama-GENBA-10B: A Trilingual Large Language Model for German, English and Bavarian

Michael Hoffmann, Jophin John, Stefan Schweter, Gokul Ramakrishnan, Hoi-Fong Mak, Alice Zhang, Dmitry Gaynullin, Nicolay J. Hammer

机构 * Leibniz Supercomputing Centre (LRZ)（莱布尼茨超算中心） ； Cerebras Systems（Cerebras系统） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；foundation model(abstract)；pretraining(abstract)

Comments Michael Hoffmann and Jophin John contributed equally to this work

Natural Context Drift Undermines the Natural Language Understanding of Large Language Models

Yulong Wu, Viktor Schlegel, Riza Batista-Navarro

机构 * Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系） ； Imperial College London, Imperial Global Singapore（伦敦帝国学院，帝国全球新加坡） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments EMNLP 2025 Findings

Assessing Social Alignment: Do Personality-Prompted Large Language Models Behave Like Humans?

Ivan Zakazov, Mikolaj Boronski, Lorenzo Drudi, Robert West

机构 * EPFL（苏黎世联邦理工学院） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Accepted to NeurIPS 2024 Workshop on Behavioral Machine Learning

Scatter-Based Innovation Propagation in Large Language Models for Multi-Stage Process Adaptation

Hong Su

机构 * School of Computer Science, Chengdu University of Information Technology（计算机科学学院，成都信息科技大學） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models

Jiawei Lian, Jianhong Pan, Lefan Wang, Yi Wang, Shaohui Mei, Lap-Pui Chau

机构 * Department of Electrical and Electronic Engineering（电子与电气工程系） ； The Hong Kong Polytechnic University（香港理工大学） ； School of Electronics and Information（电子与信息学院） ； Northwestern Polytechnical University（西北工业大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；instruction tuning(abstract)；pretraining(abstract)

PersLLM: A Personified Training Approach for Large Language Models

Zheni Zeng, Jiayi Chen, Huimin Chen, Yukun Yan, Yuxuan Chen, Zhenghao Liu, Zhiyuan Liu, Maosong Sun

机构 * Tsinghua University（清华大学） ； Northeastern University（东北大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 8 pages for main text, 5 figures

Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs

Yichun Yin, Wenyong Huang, Kaikai Song, Yehui Tang, Xueyu Wu, Wei Guo, Peng Guo, Yaoyuan Wang, Xiaojun Meng, Yasheng Wang, Dong Li, Can Chen, Dandan Tu, Yin Li, Fisher Yu, Ruiming Tang, Yunhe Wang, Baojun Wang, Bin Wang, Bo Wang, Boxiao Liu, Changzheng Zhang, Duyu Tang, Fei Mi, Hui Jin, Jiansheng Wei, Jiarui Qin, Jinpeng Li, Jun Zhao, Liqun Deng, Lin Li, Minghui Xu, Naifu Zhang, Nianzu Zheng, Qiang Li, Rongju Ruan, Shengjun Cheng, Tianyu Guo, Wei He, Wei Li, Weiwen Liu, Wulong Liu, Xinyi Dai, Yonghan Dong, Yu Pan, Yue Li, Yufei Wang, Yujun Li, Yunsheng Ni, Zhe Liu, Zhenhe Zhang, Zhicheng Liu

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Comments fix conflicts of latex pacakges

Generative adversarial networks vs large language models: a comparative study on synthetic tabular data generation

Austin A. Barr, Robert Rozman, Eddie Guo

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 12 pages, 7 figures, 5 tables

GPT-4o as the Gold Standard: A Scalable and General Purpose Approach to Filter Language Model Pretraining Data

Jifan Zhang, Ziyue Luo, Jia Liu, Ness Shroff, Robert Nowak

机构 * University of Wisconsin-Madison（威斯康星大学麦迪逊分校） ； Ohio State University, Columbus（俄亥俄州立大学哥伦布分校） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(title,abstract)；LLM(abstract)；large language model(abstract)

Trojan Detection Through Pattern Recognition for Large Language Models

Vedant Bhasin, Matthew Yudin, Razvan Stefanescu, Rauf Izmailov

机构 * Peraton Labs（佩拉顿实验室） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；RLHF(abstract)

Comments 20 pages, 11 Figures

Adaptive Few-shot Prompting for Machine Translation with Pre-trained Language Models

Lei Tang, Jinghui Qin, Wenxuan Ye, Hao Tan, Zhijing Yang

专题命中 预训练与数据 ：language model(title,abstract)；prompting(title,abstract)；LLM(abstract)；large language model(abstract)

Comments published to AAAI2025

Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models

Chengkai Huang, Yu Xia, Rui Wang, Kaige Xie, Tong Yu, Julian McAuley, Lina Yao

机构 * University of New South Wales（新南威尔士大学） ； University of California San Diego（加利福尼亚大学圣迭戈分校） ； Duke University（杜克大学） ； Georgia Institute of Technology（佐治亚理工学院） ； Adobe Research（奥多比研究院） ； CSIRO’s Data61（联邦科学与工业研究组织Data61） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Scaling Laws for Post Training Quantized Large Language Models

Zifei Xu, Alexander Lan, Wanzin Yazar, Tristan Webb, Sayeh Sharify, Xin Wang

机构 * d-Matrix（d-Matrix公司） ； Yale University（耶鲁大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；post-training(abstract)

Measuring Non-Adversarial Reproduction of Training Data in Large Language Models

Michael Aerni, Javier Rando, Edoardo Debenedetti, Nicholas Carlini, Daphne Ippolito, Florian Tramèr

机构 * ETH Zurich（苏黎世联邦理工学院） ； Google DeepMind（谷歌DeepMind） ； Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)；prompting(abstract)

LLM4Causal: Democratized Causal Tools for Everyone via Large Language Model

Haitao Jiang, Lin Ge, Yuhe Gao, Jianian Wang, Rui Song

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments Accepted by COLM2024

BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baseline

Guosheng Dong, Da Pan, Yiding Sun, Shusen Zhang, Zheng Liang, Xin Wu, Yanjun Shen, Fan Yang, Haoze Sun, Tianpeng Li, Mingan Lin, Jianhua Xu, Yufan Zhang, Xiaonan Nie, Lei Su, Bingning Wang, Wentao Zhang, Jiaxin Mao, Zenan Zhou, Weipeng Chen

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments 19 pages, 6 figures

CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition

Yafeng Zhang, Zilan Yu, Yuang Huang, Jing Tang

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)

Comments 27TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE

Pandora's White-Box: Precise Training Data Detection and Extraction in Large Language Models

Jeffrey G. Wang, Jason Wang, Marvin Li, Seth Neel

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Found software bug in experiments, withdrawing in order to address and update results

Spatial-Temporal Large Language Model for Traffic Prediction

Chenxi Liu, Sun Yang, Qianxiong Xu, Zhishuai Li, Cheng Long, Ziyue Li, Rui Zhao

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Accepted by MDM 2024 (Research Track)

Generative Monoculture in Large Language Models

Fan Wu, Emily Black, Varun Chandrasekaran

专题命中 预训练与数据 ：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；prompting(abstract)