语言大模型 / LLM

StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimization

Ziliang Wang, Xuhui Zheng, Kang An, Cijun Ouyang, Jialu Cai, Yuhang Wang, Yichao Wu

机构 * SenseTime（深睿科技） ； Nanjing University（南京大学） ； Shenzhen University（深圳大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 20 pages, 6 figures

Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling

Hongzhi Huang, Defa Zhu, Banggu Wu, Yutao Zeng, Ya Wang, Qiyang Min, Xun Zhou

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments accepted by ICML2025

InSTA: Towards Internet-Scale Training For Agents

Brandon Trabucco, Gunnar Sigurdsson, Robinson Piramuthu, Ruslan Salakhutdinov

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Amazon（亚马逊） 纠错

专题命中 预训练与数据 ：LLM(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Improved results, zero-shot transfer to Web Voyager

Scaling Law for Quantization-Aware Training

Mengzhao Chen, Chaoyi Zhang, Jing Liu, Yutao Zeng, Zeyue Xue, Zhiheng Liu, Yunshui Li, Jin Ma, Jie Huang, Xun Zhou, Ping Luo

机构 * The University of Hong Kong（香港大学） ； ByteDance Seed（字节跳动种子） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments A unified scaling law for QAT that models quantization error as a function of model size, training data volume, and quantization group size

Model Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Laws

Xiyuan Wei, Ming Lin, Fanjiang Ye, Fengguang Song, Liangliang Cao, My T. Thai, Tianbao Yang

专题命中 预训练与数据 ：foundation model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 6 figures

Towards DS-NER: Unveiling and Addressing Latent Noise in Distant Annotations

Yuyang Ding, Dan Qiao, Juntao Li, Jiajie Xu, Pingfu Chao, Xiaofang Zhou, Min Zhang

机构 * Soochow University（苏州大学） ； Hong Kong University of Science and Technology（香港理工大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

From Text to Network: Constructing a Knowledge Graph of Taiwan-Based China Studies Using Generative AI

Hsuan-Lei Shao

机构 * Graduate Institute of Health and Biotechnology Law, Taipei Medical University（健康与生物技术法研究生院，台北医学大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 4 pages, 4 figures

TSINR: Capturing Temporal Continuity via Implicit Neural Representations for Time Series Anomaly Detection

Mengxuan Li, Ke Liu, Hongyang Chen, Jiajun Bu, Hongwei Wang, Haishuai Wang

机构 * Zhejiang University（浙江大学） ； Zhejiang Lab（浙江实验室） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted by SIGKDD 2025

Generalization Capability for Imitation Learning

Yixiao Wang

机构 * University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 预训练与数据 ：language model(abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

DR.FIX: Automatically Fixing Data Races at Industry Scale

Farnaz Behrang, Zhizhou Zhang, Georgian-Vlad Saioc, Peng Liu, Milind Chabbi

机构 * Uber Technologies（Uber技术公司） ； Aarhus University（奥胡斯大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments To appear in PLDI 2025

Regional Tiny Stories: Using Small Models to Compare Language Learning and Tokenizer Performance

Nirvan Patil, Malhar Abhay Inamdar, Agnivo Gosai, Guruprasad Pathak, Anish Joshi, Aryan Sagavekar, Anish Joshirao, Raj Dandekar, Rajat Dandekar, Sreedath Panat

专题命中 预训练与数据 ：language model(abstract)；small language model(abstract)；分类 cs.CL、cs.AI

Comments 34 pages, 24 figures, 16 tables

FedEGG: Federated Learning with Explicit Global Guidance

Kun Zhai, Yifeng Gao, Difan Zou, Guangnan Ye, Siheng Chen, Xingjun Ma, Yu-Gang Jiang

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

VideoComp: Advancing Fine-Grained Compositional and Temporal Alignment in Video-Text Models

Dahun Kim, AJ Piergiovanni, Ganesh Mallya, Anelia Angelova

机构 * Google DeepMind（谷歌DeepMind） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments CVPR 2025, project page at https://github.com/google-deepmind/video_comp

Cyborg Data: Merging Human with AI Generated Training Data

Kai North, Christopher Ormerod

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Keyword-Oriented Multimodal Modeling for Euphemism Identification

Yuxue Hu, Junsong Li, Meixuan Chen, Dongyu Su, Tongguan Wang, Ying Sha

机构 * College of Informatics, Huazhong Agricultural University（华中农业大学信息学院） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

iGAiVA: Integrated Generative AI and Visual Analytics in a Machine Learning Workflow for Text Classification

Yuanzhe Jin, Adrian Carrasco-Revilla, Min Chen

机构 * University of Oxford（牛津大学） ； Inetum Spain（Inetum西班牙公司） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

A Comparative Analysis of Word Segmentation, Part-of-Speech Tagging, and Named Entity Recognition for Historical Chinese Sources, 1900-1950

Zhao Fang, Liang-Chun Wu, Xuening Kong, Spencer Dean Stewart

机构 * University of Chicago（芝加哥大学） ； Purdue University（普渡大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to NLP4DH 2025 at NAACL 2025

Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning

Tian Liu, Huixin Zhang, Shubham Parashar, Shu Kong

机构 * Texas A&M University（德克萨斯农工大学） ； University of Macau（澳门大学） ； Institute of Collaborative Innovation（协同创新研究院） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.AI、cs.LG

Comments Accepted to CVPR 2025. Website and code: https://tian1327.github.io/SWAT/

HAR-DoReMi: Optimizing Data Mixture for Self-Supervised Human Activity Recognition Across Heterogeneous IMU Datasets

Lulu Ban, Tao Zhu, Xiangqing Lu, Qi Qiu, Wenyong Han, Shuangjian Li, Liming Chen, Kevin I-Kai Wang, Mingxing Nie, Yaping Wan

机构 * School of Computer Science, University of South China（南华大学计算机学院） ； School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院） ； Engineering and Design, Department of Electrical, Computer and Software Engineering, University of Auckland（奥克兰大学电气、计算机与软件工程学院工程设计部门） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

A Framework to Assess Multilingual Vulnerabilities of LLMs

Likai Tang, Niruth Bogahawatta, Yasod Ginige, Jiarui Xu, Shixuan Sun, Surangika Ranathunga, Suranga Seneviratne

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

A Recipe for Improving Remote Sensing VLM Zero Shot Generalization

Aviad Barzilai, Yotam Gigi, Amr Helmy, Vered Silverman, Yehonathan Refael, Bolous Jaber, Tomer Shekel, George Leifman, Genady Beryozkin

机构 * Google Research（谷歌研究院） 纠错

专题命中 预训练与数据 ：language model(abstract)；foundation model(abstract)；分类 cs.AI、cs.LG

TexAVi: Generating Stereoscopic VR Video Clips from Text Descriptions

Vriksha Srihari, R. Bhavya, Shruti Jayaraman, V. Mary Anita Rajam

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Co-authors do not consent to publishing on Arxiv

Journal ref TexAVi: Generating Stereoscopic VR Video Clips from Text Descriptions, 2024 IEEE International Conference on Computer Vision and Machine Intelligence (CVMI), Prayagraj, India, 2024, pp. 1-6

E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions

Hongbo Zheng, Suyuan Wang, Neeraj Gangwar, Nickvash Kani

机构 * University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Provable Benefits of Task-Specific Prompts for In-context Learning

Xiangyu Chang, Yingcong Li, Muti Kara, Samet Oymak, Amit K. Roy-Chowdhury

机构 * University of California, Riverside（加州大学河滨分校） ； University of Michigan（密歇根大学） ； Bilkent University（比尔肯特大学） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments Proceedings of the 28th International Conference on Artificial Intelligence and Statistics (AISTATS) 2025

Q-NL Verifier: Leveraging Synthetic Data for Robust Knowledge Graph Question Answering

Tim Schwabe, Louisa Siebel, Patrik Valach, Maribel Acosta

机构 * Technical University of Munich（慕尼黑工业大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches

Yifang Chen, Xuyang Guo, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song

机构 * The University of Chicago（芝加哥大学） ； Guilin University of Electronic Technology（桂林电子科技大学） ； University of New South Wales（新南威尔士大学） ； The University of Hong Kong（香港大学） ； University of Wisconsin-Madison（威斯康星大学麦迪逊分校） ； The Simons Institute for the Theory of Computing at UC Berkeley（加州大学伯克利分校西蒙斯理论计算研究所） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents

Shi Yu, Chaoyue Tang, Bokai Xu, Junbo Cui, Junhao Ran, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

机构 * Tsinghua University（清华大学） ； ModelBest Inc.（模贝斯特公司） ； Rice University（莱斯大学） ； Northeastern University（东北大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting

Suraj Anand, Michael A. Lepori, Jack Merullo, Ellie Pavlick

机构 * Brown University（布朗大学） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

Comments 10 pages, 6 figures

Deterministic or probabilistic? The psychology of LLMs as random number generators

Javier Coronado-Blázquez

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 31 pages, 12 figures

Political Events using RAG with LLMs

Muhammad Arslan, Saba Munawar, Christophe Cruz

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI