语言大模型 / LLM

VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment

Yogesh Kulkarni, Pooyan Fazli

机构 * Arizona State University（亚利桑那州立大学） 纠错

专题命中 预训练与数据 ：LLM(title)；language model(abstract)；pretraining(abstract)；preference optimization(abstract)

Comments EMNLP 2025 (Main)

Hopscotch: Discovering and Skipping Redundancies in Language Models

Mustafa Eyceoz, Nikhil Shivakumar Nayak, Hao Wang, Ligong Han, Akash Srivastava

机构 * Red Hat AI Innovation（红帽AI创新） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 10 pages, 4 figures, 9 tables

Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Models

Thao Nguyen, Yang Li, Olga Golovneva, Luke Zettlemoyer, Sewoong Oh, Ludwig Schmidt, Xian Li

机构 * FAIR at Meta（Meta 的 FAIR 研究所） ； University of Washington（华盛顿大学） ； Stanford University（斯坦福大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to COLM 2025

AutoVeriFix: Automatically Correcting Errors and Enhancing Functional Correctness in LLM-Generated Verilog Code

Yan Tan, Xiangchen Meng, Zijun Jiang, Yangdi Lyu

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Synthetic vs. Gold: The Role of LLM Generated Labels and Data in Cyberbullying Detection

Arefeh Kazemi, Sri Balaaji Natarajan Kalaivendan, Joachim Wagner, Hamza Qadeer, Kanishk Verma, Brian Davis

机构 * School of Computing, ADAPT Centre, Dublin City University, Dublin, Ireland（计算学院、ADAPT中心、都柏林城市大学、都柏林、爱尔兰） 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

ToolACE: Winning the Points of LLM Function Calling

Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong Wang, Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Xinzhi Wang, Yong Liu, Yasheng Wang, Duyu Tang, Dandan Tu, Lifeng Shang, Xin Jiang, Ruiming Tang, Defu Lian, Qun Liu, Enhong Chen

机构 * Shanghai Jiao Tong University（上海交通大学） ； University of Science and Technology of China（中国科学技术大学） ； Huawei Noah’s Ark Lab（华为诺亚实验室） ； Huawei Technologies Co., Ltd（华为技术有限公司） ； Tsinghua University（清华大学） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 21 pages, 22 figures

TR-LLM: Integrating Trajectory Data for Scene-Aware LLM-Based Human Action Prediction

Kojiro Takeyama, Yimeng Liu, Misha Sra

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments Accepted to IROS 2025

Random Initialization Can't Catch Up: The Advantage of Language Model Transfer for Time Series Forecasting

Roland Riachi, Kashif Rasul, Arjun Ashok, Prateek Humane, Alexis Roger, Andrew R. Williams, Yuriy Nevmyvaka, Irina Rish

机构 * University of Montreal（蒙特利尔大学） ； Mila - Québec AI Institute（魁北克人工智能研究所） ； McGill University（麦吉尔大学） 纠错

专题命中 预训练与数据 ：language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.AI、cs.LG

Assembly of Experts: Linear-time construction of the Chimera LLM variants with emergent and adaptable behaviors

Henrik Klagges, Robert Dahlke, Fabian Klemm, Benjamin Merkel, Daniel Klingmann, David A. Reiss, Dan Zecha

机构 * TNG Technology Consulting GmbH（TNG技术咨询公司） 纠错

专题命中 预训练与数据 ：LLM(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases

Michael Y. Hu, Jackson Petty, Chuan Shi, William Merrill, Tal Linzen

机构 * Center for Data Science（数据科学中心） ； Department of Linguistics（语言学系） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2025 Camera Ready

Physics of Language Models: Part 1, Learning Hierarchical Language Structures

Zeyuan Allen-Zhu, Yuanzhi Li

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments V2 polishes writing and adds Appendix G; V3 polishes writing and changes the title; V4 improves writing and adds Appendix H (more uniform attention results)

MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment

Siyuan Yan, Xieji Li, Ming Hu, Yiwen Jiang, Zhen Yu, Zongyuan Ge

机构 * Faculty of Engineering, Monash University, Melbourne, Australia（墨尔本大学工程学院） ； AIM for Health Lab, Monash University, Victoria, Australia（墨尔本大学健康人工智能实验室） 纠错

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)

Comments MICCAI2025 early acceptance; First two authors contribute equally

Impact of Noisy Supervision in Foundation Model Learning

Hao Chen, Zihan Wang, Ran Tao, Hongxin Wei, Xing Xie, Masashi Sugiyama, Bhiksha Raj, Jindong Wang

专题命中 预训练与数据 ：foundation model(title,abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 18 pages, 10 figures, 6 tables, preprint. arXiv admin note: substantial text overlap with arXiv:2309.17002

Enhancing Leakage Attacks on Searchable Symmetric Encryption Using LLM-Based Synthetic Data Generation

Joshua Chiu, Partha Protim Paul, Zahin Wahab

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens

Kaihang Pan, Wang Lin, Zhongqi Yue, Tenglong Ao, Liyu Jia, Wei Zhao, Juncheng Li, Siliang Tang, Hanwang Zhang

机构 * Zhejiang University（浙江大学） ； Nanyang Technological University（新加坡国立大学） ； Peking University（北京大学） ； Huawei Singapore Research Center（华为新加坡研究中心） 纠错

专题命中 预训练与数据 ：pretraining(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted by CVPR 2025 (Oral)

Liquid: Language Models are Scalable and Unified Multi-modal Generators

Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, Xiang Bai

专题命中 预训练与数据 ：language model(title,abstract)；LLM(abstract)；large language model(abstract)

Comments Technical report. Project page: https://foundationvision.github.io/Liquid/

Is Temporal Prompting All We Need For Limited Labeled Action Recognition?

Shreyank N Gowda, Boyan Gao, Xiao Gu, Xiaobo Jin

专题命中 预训练与数据 ：prompting(title,abstract)；language model(abstract)；pretraining(abstract)

Comments Accepted in CVPR-W 2025

LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning

Zhe Li, Weihao Yuan, Yisheng He, Lingteng Qiu, Shenhao Zhu, Xiaodong Gu, Weichao Shen, Yuan Dong, Zilong Dong, Laurence T. Yang

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)

Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarization

Yen-Ju Lu, Ting-Yao Hu, Hema Swetha Koppula, Hadi Pouransari, Jen-Hao Rick Chang, Yin Xia, Xiang Kong, Qi Zhu, Simon Wang, Oncel Tuzel, Raviteja Vemulapalli

专题命中 预训练与数据 ：LLM(title,abstract)；preference optimization(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL 2025 Findings

Do Egocentric Video-Language Models Truly Understand Hand-Object Interactions?

Boshen Xu, Ziheng Wang, Yang Du, Zhinan Song, Sipeng Zheng, Qin Jin

专题命中 预训练与数据 ：language model(title,abstract)；large language model(abstract)；pretraining(abstract)

Comments Accepted by ICLR 2025. Code: https://github.com/xuboshen/EgoNCEpp

Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models

Pit Neitemeier, Björn Deiseroth, Constantin Eichenberg, Lukas Balles

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Towards Best Practices for Open Datasets for LLM Training

Stefan Baack, Stella Biderman, Kasia Odrozek, Aviya Skowron, Ayah Bdeir, Jillian Bommarito, Jennifer Ding, Maximilian Gahntz, Paul Keller, Pierre-Carl Langlais, Greg Lindahl, Sebastian Majstorovic, Nik Marda, Guilherme Penedo, Maarten Van Segbroeck, Jennifer Wang, Leandro von Werra, Mitchell Baker, Julie Belião, Kasia Chmielinski, Marzieh Fadaee, Lisa Gutermuth, Hynek Kydlíček, Greg Leppert, EM Lewis-Jong, Solana Larsen, Shayne Longpre, Angela Oduor Lungati, Cullen Miller, Victor Miller, Max Ryabinin, Kathleen Siminyu, Andrew Strait, Mark Surman, Anna Tumadóttir, Maurice Weber, Rebecca Weiss, Lee White, Thomas Wolf

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Enhancing Remote Sensing Vision-Language Models for Zero-Shot Scene Classification

Karim El Khoury, Maxime Zanella, Benoît Gérin, Tiffanie Godelaine, Benoît Macq, Saïd Mahmoudi, Christophe De Vleeschouwer, Ismail Ben Ayed

专题命中 预训练与数据 ：language model(title,abstract)；pretraining(abstract)；prompting(abstract)

Comments Accepted at ICASSP 2025

METAGENE-1: Metagenomic Foundation Model for Pandemic Monitoring

Ollie Liu, Sami Jaghouar, Johannes Hagemann, Shangshang Wang, Jason Wiemels, Jeff Kaufman, Willie Neiswanger

专题命中 预训练与数据 ：foundation model(title,abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs

Ioannis Tzachristas

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments in Greek language. This Master's Thesis was supervised by Prof. Nikolaos Papaspyrou (National Technical University of Athens) and Dr. Aifen Sui (Huawei Munich Research Center). English version: pages 57-104. Original submission link: http://artemis.cslab.ece.ntua.gr:8080/jspui/handle/123456789/19180

Properties and Challenges of LLM-Generated Explanations

Jenny Kunz, Marco Kuhlmann

专题命中 预训练与数据 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

UniMed-CLIP: Towards a Unified Image-Text Pretraining Paradigm for Diverse Medical Imaging Modalities

Muhammad Uzair Khattak, Shahina Kunhimon, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan

专题命中 预训练与数据 ：pretraining(title,abstract)；large language model(abstract)；language model(abstract)

Comments Code, models and demo available at https://github.com/mbzuai-oryx/UniMed-CLIP

Unleashing GHOST: An LLM-Powered Framework for Automated Hardware Trojan Design

Md Omar Faruque, Peter Jamieson, Ahmad Patooghy, Abdel-Hameed A. Badawy

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

DuetML: Human-LLM Collaborative Machine Learning Framework for Non-Expert Users

Wataru Kawabe, Yusuke Sugano

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments 22 pages, 10 figures

I'm Spartacus, No, I'm Spartacus: Measuring and Understanding LLM Identity Confusion

Kun Li, Shichao Zhuang, Yue Zhang, Minghui Xu, Ruoxi Wang, Kaidi Xu, Xinwen Fu, Xiuzhen Cheng

专题命中 预训练与数据 ：LLM(title,abstract)；large language model(abstract)；language model(abstract)

Comments 16 pages, 8 figure, 6 tables