语言大模型 / LLM

CHRONOBERG: Capturing Language Evolution and Temporal Awareness in Foundation Models

Niharika Hegde, Subarnaduti Paul, Lars Joel-Frey, Manuel Brack, Kristian Kersting, Martin Mundt, Patrick Schramowski

机构 * German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心） ； Department of Computer Science and Mathematics, University of Bremen（不莱梅大学计算机科学与数学系） ； Hessian Center for AI (hessian.AI)（黑森人工智能中心） ； Department of Computer Science, TU Darmstadt（图宾根大学计算机科学系） ； Centre for Cognitive Science, TU Darmstadt（图宾根大学认知科学中心） ； CERTAIN 纠错

专题命中 预训练与数据 ：foundation model(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Verification Limits Code LLM Training

Srishti Gureja, Elena Tommasone, Jingyi He, Sara Hooker, Matthias Gallé, Marzieh Fadaee

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Machine-generated text detection prevents language model collapse

George Drayson, Emine Yilmaz, Vasileios Lampos

机构 * Centre for Artificial Intelligence（人工智能中心） ； Department of Computer Science（计算机科学系） ； University College London（伦敦大学学院） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

MountainLion: A Multi-Modal LLM-Based Agent System for Interpretable and Adaptive Financial Trading

Siyi Wu, Junqiao Wang, Zhaoyang Guan, Leyi Zhao, Xinyuan Song, Xinyu Ying, Dexu Yu, Jinhao Wang, Hanlin Zhang, Michele Pak, Yangfan He, Yi Xin, Jianhui Wang, Tianyu Shi

机构 * The University of Texas at Arlington（德克萨斯大学阿灵顿分校） ； Sichuan University（四川大学） ； Northwestern University（西北大学） ； Indiana University（印第安纳大学） ； Emory University（埃默里大学） ； Nankai University（南开大学） ； MountainLion Research（MountainLion研究机构） ； Xi’an University of Electronic Science and Technology（西安电子科技大学） ； Kyoto University（京都大学） ； University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校） ； Nanjing university（南京大学） ； Tsinghua University（清华大学） ； University of Toronto（多伦多大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Subjective Behaviors and Preferences in LLM: Language of Browsing

Sai Sundaresan, Harshita Chopra, Atanu R. Sinha, Koustava Goswami, Nagasai Saketh Naidu, Raghav Karan, N Anushka

机构 * Adobe Research（Adobe研究院） ； University of Washington, Seattle（华盛顿大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP 2025

Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Models

Zheyu Zhang, Shuo Yang, Bardh Prenkaj, Gjergji Kasneci

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to EMNLP 2025 (Findings)

TAGAL: Tabular Data Generation using Agentic LLM Methods

Benoît Ronval, Pierre Dupont, Siegfried Nijssen

机构 * UCLouvain（乌尔特里大学） ； ICTEAM（信息与团队研究所） ； KU Leuven（根特大学） ； DTAI（数据与人工智能研究所） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation

Jinyi Han, Tingyun Li, Shisong Chen, Jie Shi, Xinyi Wang, Guanglei Yue, Jiaqing Liang, Xin Lin, Liqian Wen, Zulong Chen, Yanghua Xiao

机构 * Shanghai Institute of Artificial Intelligence for Education（上海人工智能教育研究院） ； East China Normal University（华东师范大学） ； School of Data Science, Fudan University（复旦大学数据科学学院） ； College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院） ； Alibaba（阿里巴巴集团） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments The initial versin was made in August 2024

Event-Aware Sentiment Factors from LLM-Augmented Financial Tweets: A Transparent Framework for Interpretable Quant Trading

Yueyi Wang, Qiyao Wei

机构 * Department of Physics, University of Cambridge, Cambridge, UK（剑桥大学物理系） ； Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK（剑桥大学应用数学与理论物理系） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 16 pages, 12 figures, accepted at ICML 2025 New in ML Workshop

Topic Over Source: The Key to Effective Data Mixing for Language Models Pre-training

Jiahui Peng, Xinlin Zhuang, Jiantao Qiu, Ren Ma, Jing Yu, He Zhu, Conghui He

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage

Yuzhou Nie, Zhun Wang, Ye Yu, Xian Wu, Xuandong Zhao, Wenbo Guo, Dawn Song

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted by COLM 2025

Kronos: A Foundation Model for the Language of Financial Markets

Yu Shi, Zongliang Fu, Shuo Chen, Bohan Zhao, Wei Xu, Changshui Zhang, Jian Li

专题命中 预训练与数据 ：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge

Xinyue Cui, Johnny Tian-Zheng Wei, Swabha Swayamdipta, Robin Jia

机构 * University of Southern California（南加州大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ACL 2025 Findings

GRR-CoCa: Leveraging LLM Mechanisms in Multimodal Model Architectures

Jake R. Patock, Nicole Catherine Lewis, Kevin McCoy, Christina Gomez, Canling Chen, Lorenzo Luzi

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 12 pages, 2 figures

SynthCTI: LLM-Driven Synthetic CTI Generation to enhance MITRE Technique Mapping

Álvaro Ruiz-Ródenas, Jaime Pujante Sáez, Daniel García-Algora, Mario Rodríguez Béjar, Jorge Blasco, José Luis Hernández-Ramos

机构 * Department of Information and Communication Engineering, Universidad de Murcia（信息与通信工程系，穆尔西亚大学） ； Department of Computer Systems, Universidad Politécnica de Madrid（计算机系统系，马德里理工大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 17 pages, 13 figures

LLM Data Selection and Utilization via Dynamic Bi-level Optimization

Yang Yu, Kai Han, Hang Zhou, Yehui Tang, Kaiqi Huang, Yunhe Wang, Dacheng Tao

机构 * School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所认知与决策智能复杂系统重点实验室） ； Huawei Noah’s Ark Lab（华为诺亚实验室） ； College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments The 42nd International Conference on Machine Learning (ICML 2025)

Two-Stage Pretraining for Molecular Property Prediction in the Wild

Kevin Tirta Wijaya, Minghao Guo, Michael Sun, Hans-Peter Seidel, Wojciech Matusik, Vahid Babaei

机构 * Max Planck Institute for Informatics（马克斯·普朗克研究所） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

DataDecide: How to Predict Best Pretraining Data with Small Experiments

Ian Magnusson, Nguyen Tai, Ben Bogin, David Heineman, Jena D. Hwang, Luca Soldaini, Akshita Bhagia, Jiacheng Liu, Dirk Groeneveld, Oyvind Tafjord, Noah A. Smith, Pang Wei Koh, Jesse Dodge

机构 * Allen Institute for AI ； Paul G. Allen School of Computer Science \& Engineering, University of Washington ； University of Pennsylvania 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments ICML 2025

Leveraging LLM-Assisted Query Understanding for Live Retrieval-Augmented Generation

Guanting Dong, Xiaoxi Li, Yuyao Zhang, Mengjie Deng

机构 * Renmin University of China（中国人民大学） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at SIGIR 2025 LiveRAG Workshop (Oral Presentation)

OAT-Rephrase: Optimization-Aware Training Data Rephrasing for Zeroth-Order LLM Fine-Tuning

Jikai Long, Zijian Hu, Xiaodong Yu, Jianwen Xie, Zhaozhuo Xu

机构 * Stevens Institute of Technology（史蒂文斯理工学院） ； Lambda, Inc.（Lambda公司） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback

Yongjin Yang, Sinjae Kang, Juyong Lee, Dongjun Lee, Se-Young Yun, Kimin Lee

专题命中 预训练与数据 ：language agent(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Preprint, under review

Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model

Chong Li, Yingzhuo Deng, Jiajun Zhang, Chengqing Zong

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS, Beijing, China（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院，北京） ； School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China（人工智能学院，中国科学院大学，北京） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments ACL 2025, our codes and models are available at https://github.com/ZNLP/DMoE

Scaling Laws for Upcycling Mixture-of-Experts Language Models

Seng Pei Liew, Takuya Kato, Sho Takase

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments ICML 2025. 16 figures, 8 tables. Code available at https://github.com/sbintuitions/sparse-upcycling-scaling-laws

Relational Schemata in BERT Are Inducible, Not Emergent: A Study of Performance vs. Competence in Language Models

Cole Gawin

机构 * Cole Gawin（独立研究者） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 4 figures, 3 tables

A Scaling Law for Token Efficiency in LLM Fine-Tuning Under Fixed Compute Budgets

Ryan Lagasse, Aidan Kierans, Avijit Ghosh, Shiri Dori-Hacohen

机构 * University of Connecticut Storrs（康涅狄格大学斯托尔斯分校） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining

Wenqi Zhang, Hang Zhang, Xin Li, Jiashuo Sun, Yongliang Shen, Weiming Lu, Deli Zhao, Yueting Zhuang, Lidong Bing

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； DAMO Academy, Alibaba Group（阿里巴巴集团达摩院） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；LLM(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Under review

Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis

Albérick Euraste Djiré, Abdoul Kader Kaboré, Earl T. Barr, Jacques Klein, Tegawendé F. Bissyandé

机构 * ucl.ac.uk（伦敦大学学院） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

MIND: Math Informed syNthetic Dialogues for Pretraining LLMs

Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu, Sanjeev Satheesh, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 31 pages, 5 figures, 14 tables

Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws

Nikhil Sardana, Jacob Portes, Sasha Doubov, Jonathan Frankle

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments 16 pages, 7 figures, In the 41st International Conference on Machine Learning, 2024

CroissantLLM: A Truly Bilingual French-English Language Model

Manuel Faysse, Patrick Fernandes, Nuno M. Guerreiro, António Loison, Duarte M. Alves, Caio Corro, Nicolas Boizard, João Alves, Ricardo Rei, Pedro H. Martins, Antoni Bigata Casademunt, François Yvon, André F. T. Martins, Gautier Viaud, Céline Hudelot, Pierre Colombo

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG