语言大模型 / LLM

VulCoCo: A Simple Yet Effective Method for Detecting Vulnerable Code Clones

Tan Bui, Yan Naing Tun, Thanh Phuc Nguyen, Yindu Su, Ferdian Thung, Yikun Li, Han Wei Ang, Yide Yin, Frank Liauw, Lwin Khin Shar, Eng Lieh Ouh, Ting Zhang, David Lo

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts

Yifan Zhang, Yifan Luo, Yang Yuan, Andrew C Yao

机构 * Tsinghua University（清华大学） ； Shanghai Qi Zhi Institute（上海启智研究院） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in ACL 2025 Findings

Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by Learning Language-Agnostic Speech Representations

Jeong Hun Yeo, Minsu Kim, Chae Won Kim, Stavros Petridis, Yong Man Ro

机构 * KAIST（韩国科学技术院） ； Imperial College London（伦敦帝国理工学院） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Accepted at ICCV 2025. Code available at: https://github.com/JeongHun0716/zero-avsr

Cube: A Roblox View of 3D Intelligence

Foundation AI Team, Kiran Bhat, Nishchaie Khanna, Karun Channa, Tinghui Zhou, Yiheng Zhu, Xiaoxia Sun, Charles Shang, Anirudh Sudarshan, Maurice Chu, Daiqing Li, Kangle Deng, Jean-Philippe Fauconnier, Tijmen Verhulsdonck, Maneesh Agrawala, Kayvon Fatahalian, Alexander Weiss, Christian Reiser, Ravi Kiran Chirravuri, Ravali Kandur, Alejandro Pelaez, Akash Garg, Michael Palleschi, Jessica Wang, Skylar Litz, Leon Liu, Anying Li, David Harmon, Derek Liu, Liangjun Feng, Denis Goupil, Lukas Kuczynski, Jihyun Yoon, Naveen Marri, Peiye Zhuang, Yinan Zhang, Brian Yin, Haomiao Jiang, Marcel van Workum, Thomas Lane, Bryce Erickson, Salil Pathare, Kyle Price, Steve Han, Yiqing Wang, Anupam Singh, David Baszucki

机构 * Foundation AI Team（基础人工智能团队） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Comments Our code and model weights can be found at: https://github.com/Roblox/cube

FinRL Contests: Benchmarking Data-driven Financial Reinforcement Learning Agents

Keyi Wang, Nikolaus Holzer, Ziyi Xia, Yupeng Cao, Jiechao Gao, Anwar Walid, Kairong Xiao, Xiao-Yang Liu Yanglet

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments arXiv admin note: text overlap with arXiv:2501.10709

Can AI Rely on the Systematicity of Truth? The Challenge of Modelling Normative Domains

Matthieu Queloz

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Journal ref Philosophy and Technology 38 (34): 1-27 (2025)

Large-Language-Model Enabled Semantic Communication Systems

Zhenyi Wang, Li Zou, Shengyun Wei, Kai Li, Feifan Liao, Haibo Mi, Rongxuan Lai

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments accepted by applied sciences

Journal ref Applied Sciences. 2025; 15(13):7227

PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculum

Shiqi Zhang, Sha Zhang, Jiajun Deng, Yedong Shen, Mingxiao MA, Yanyong Zhang

机构 * University of Science and Technology of China（中国科学技术大学） ； The University of Adelaide（阿德莱德大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Talking to GDELT Through Knowledge Graphs

Audun Myers, Max Vargas, Sinan G. Aksoy, Cliff Joslyn, Benjamin Wilson, Lee Burke, Tom Grimes

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Harnessing the Power of Reinforcement Learning for Language-Model-Based Information Retriever via Query-Document Co-Augmentation

Jingming Liu, Yumeng Li, Wei Shi, Yao-Xiang Ding, Hui Su, Kun Zhou

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

How Much Can We Forget about Data Contamination?

Sebastian Bordt, Suraj Srinivas, Valentyn Boreiko, Ulrike von Luxburg

机构 * University of Tübingen, Tübingen AI Center, Germany ； Bosch Research North America \& Bosch Center for Artificial Intelligence (BCAI), Sunnyvale, USA 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2025 camera ready

Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training

Mozhi Zhang, Howe Tissue, Lu Wang, Xipeng Qiu

机构 * School of Computer Science, Fudan University, Shanghai, China（复旦大学计算机学院，上海，中国） 纠错

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ICML2025

SensorLM: Learning the Language of Wearable Sensors

Yuwei Zhang, Kumar Ayush, Siyuan Qiao, A. Ali Heydari, Girish Narayanswamy, Maxwell A. Xu, Ahmed A. Metwally, Shawn Xu, Jake Garrison, Xuhai Xu, Tim Althoff, Yun Liu, Pushmeet Kohli, Jiening Zhan, Mark Malhotra, Shwetak Patel, Cecilia Mascolo, Xin Liu, Daniel McDuff, Yuzhe Yang

机构 * Google DeepMind（谷歌DeepMind） ； Google Research（谷歌研究） ； University of Cambridge（剑桥大学） 纠错

专题命中 预训练与数据 ：foundation model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Code-Switching Curriculum Learning for Multilingual Transfer in LLMs

Haneul Yoo, Cheonbok Park, Sangdoo Yun, Alice Oh, Hwaran Lee

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To appear in Findings of ACL 2025

Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit

Charles Goddard, Fernando Fernandes Neto

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Diversity of Transformer Layers: One Aspect of Parameter Scaling Laws

Hidetaka Kamigaito, Ying Zhang, Jingun Kwon, Katsuhiko Hayashi, Manabu Okumura, Taro Watanabe

机构 * Nara Institute of Science and Technology（奈良科学技術大學） ； RIKEN Center for Advanced Intelligence Project（RIKEN先進 Intelligenceプロジェクト） ； Chungnam National University（Chungnam國立大學） ； The University of Tokyo（東京大學） ； Institute of Science Tokyo（東京科學研究院） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Language-Image Alignment with Fixed Text Encoders

Jingfeng Yang, Ziyang Wu, Yue Zhao, Yi Ma

机构 * UC Berkeley（伯克利大学） ； The University of Hong Kong（香港大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Pan-Arctic Permafrost Landform and Human-built Infrastructure Feature Detection with Vision Transformers and Location Embeddings

Amal S. Perera, David Fernandez, Chandi Witharana, Elias Manos, Michael Pimenta, Anna K. Liljedahl, Ingmar Nitze, Yili Yang, Todd Nicholson, Chia-Yu Hsu, Wenwen Li, Guido Grosse

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 20 pages, 2 column IEEE format, 13 Figures

A Hitchhiker's Guide to Scaling Law Estimation

Leshem Choshen, Yang Zhang, Jacob Andreas

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections

Da Xiao, Qingye Meng, Shengping Li, Xingyuan Yuan

专题命中 预训练与数据 ：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to the 42nd International Conference on Machine Learning (ICML'25)

FocusChat: Text-guided Long Video Understanding via Spatiotemporal Information Filtering

Zheng Cheng, Rendong Wang, Zhicheng Wang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 11 pages, 4 figures

Task-Informed Anti-Curriculum by Masking Improves Downstream Performance on Text

Andrei Jarca, Florinel Alin Croitoru, Radu Tudor Ionescu

机构 * University of Bucharest（布加勒斯特大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2025

How Good is ChatGPT in Giving Advice on Your Visualization Design?

Nam Wook Kim, Yongsu Ahn, Grace Myers, Benjamin Bach

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 34 pages, 4 figures

Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-training

Quanjiang Guo, Jinchuan Zhang, Sijie Wang, Ling Tian, Zhao Kang, Bin Yan, Weidong Xiao

机构 * University of Electronic Science and Technology of China（电子科技大学） ； Nanyang Technological University（南洋理工大学） ； Information Engineering University（信息工程大学） ； National University of Defense Technology（国防科技大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 13 pages, 6 figures, Appear on IJCAI 2025

End-to-End Vision Tokenizer Tuning

Wenxuan Wang, Fan Zhang, Yufeng Cui, Haiwen Diao, Zhuoyan Luo, Huchuan Lu, Jing Liu, Xinlong Wang

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） ； School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） ； Dalian University of Technology（大连理工大学） ； Tsinghua University（清华大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)

Bielik v3 Small: Technical Report

Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel, Adrian Gwoździej

机构 * SpeakLeash ； ACK Cyfronet AGH ； Jagiellonian University（雅盖隆大学） ； Azurro ； Enelpol 纠错

专题命中 预训练与数据 ：LLM(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Generating Traffic Scenarios via In-Context Learning to Learn Better Motion Planner

Aizierjiang Aiersilan

机构 * Aizierjiang Aiersilan（独立研究者） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments We are excited to announce that this paper has been accepted for oral presentation at the AAAI 2025 Main Conference. We are grateful for the insightful feedback from the reviewers and look forward to contributing to the discussions at AAAI

Perception Encoder: The best visual embeddings are not at the output of the network

Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Daniel Li, Piotr Dollár, Christoph Feichtenhofer

机构 * Meta FAIR ； Fudan University（复旦大学） ； Meta Reality Labs 纠错

专题命中 预训练与数据 ：LLM(abstract)；language model(abstract)；pretraining(abstract)

Comments Updated refs, fixed typos, and added new COCO SotA: 66.0 val mAP! Code, models, and data at https://github.com/facebookresearch/perception_models

3D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer

Jiajun Deng, Tianyu He, Li Jiang, Tianyu Wang, Feras Dayoub, Ian Reid

机构 * Australian Institute for Machine Learning, The University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学） ； Microsoft Research（微软研究院） ； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；instruction tuning(abstract)；pretraining(abstract)

Comments Accepted by CVPR 2025

Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similarity

Ren-Di Wu, Yu-Yen Lin, Huei-Fang Yang

机构 * National Sun Yat-sen University（国立中山大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 14 pages, 6 figures, International Conference on Technologies and Applications of Artificial Intelligence (TAAI) Camera Ready

Journal ref Technologies and Applications of Artificial Intelligence, pp. 77-90, Springer, 2025