语言大模型 / LLM

Better Aligned with Survey Respondents or Training Data? Unveiling Political Leanings of LLMs on U.S. Supreme Court Cases

Shanshan Xu, T. Y. S. S Santosh, Yanai Elazar, Quirin Vogel, Barbara Plank, Matthias Grabmair

机构 * Technical University of Munich（慕尼黑技术大学） ； Allen Institute for AI（艾伦人工智能研究所） ； University of Washington（华盛顿大学） ； LMU Munich & Munich Center for Machine Learning (MCML)（慕尼黑大学及慕尼黑机器学习中心） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Document-Level Tabular Numerical Cross-Checking: A Coarse-to-Fine Approach

Chaoxu Pang, Yixuan Cao, Ganbin Zhou, Hongwei Li, Ping Luo

机构 * Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所智能信息处理重点实验室） ； Beijing PAI Technology Ltd（北京PAI技术有限公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments Submitted to IEEE TKDE

One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers

Diana Abagyan, Alejandro R. Salamanca, Andres Felipe Cruz-Salinas, Kris Cao, Hangyu Lin, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, Sara Hooker

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；post-training(abstract)

Does Prompt Design Impact Quality of Data Imputation by LLMs?

Shreenidhi Srinivasan, Lydia Manikonda

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 7 pages

Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation

Vera Neplenbroek, Arianna Bisazza, Raquel Fernández

机构 * Institute for Logic, Language and Computation, University of Amsterdam（逻辑、语言与计算研究所，阿姆斯特丹大学） ； Center for Language and Cognition, University of Groningen（语言与认知中心，格罗宁根大学） 纠错

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；preference optimization(abstract)

Comments Accepted to the Findings of ACL 2025

Comparing Moral Values in Western English-speaking societies and LLMs with Word Associations

Chaoyi Xiang, Chunhua Liu, Simon De Deyne, Lea Frermann

机构 * School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院） ； Complex Human Data Hub, The University of Melbourne（墨尔本大学复杂人类数据中心） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments 9 pages,7 figures. Accepted to the ACL 2025 conference

Which Demographics do LLMs Default to During Annotation?

Johannes Schäfer, Aidan Combs, Christopher Bagdon, Jiahui Li, Nadine Probol, Lynn Greschner, Sean Papay, Yarik Menchaca Resendiz, Aswathy Velutharambath, Amelie Wührl, Sabine Weber, Roman Klinger

机构 * Fundamentals of Natural Language Processing, University of Bamberg, Germany（自然语言处理基础，巴姆贝格大学，德国） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments ACL 2025

The Prompt is Mightier than the Example

Shengzhe Xu, Nikhil Muralidhar, Naren Ramakrishnan

机构 * Computer Science Virginia Tech（计算机科学维吉尼亚理工大学） ； Computer Science Stevens Institute of Technology（计算机科学史蒂文斯理工学院） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworks

Xinyu Wang, Jinbo Bi, Minghu Song

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 17 pages, 5 main figures, 2 appendix figures. Submitted to ICML 2025

Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report

Paul Kassianik, Baturay Saglam, Alexander Chen, Blaine Nelson, Anu Vellore, Massimo Aufiero, Fraser Burch, Dhruv Kedia, Avi Zohary, Sajana Weerawardhena, Aman Priyanshu, Adam Swanda, Amy Chang, Hyrum Anderson, Kojin Oshiba, Omar Santos, Yaron Singer, Amin Karbasi

机构 * Foundation AI – Cisco Systems Inc.（基础AI – 奇思科技公司） ； Yale University（耶鲁大学） ； Security & Trust Organization – Cisco Systems Inc.（安全与信任组织 – 奇思科技公司） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

The Synthetic Imputation Approach: Generating Optimal Synthetic Texts For Underrepresented Categories In Supervised Classification Tasks

Joan C. Timoneda

机构 * Purdue University（普渡大学） 纠错

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

a1: Steep Test-time Scaling Law via Environment Augmented Generation

Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, Yuyao Ge, Jun Wan, Yurong Wu, Xueqi Cheng

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling

Lechao Xiao

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 25 pages

Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?

Marika Swanberg, Ryan McKenna, Edo Roth, Albert Cheu, Peter Kairouz

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；foundation model(abstract)

Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs

Tamzeed Mahfuz, Satak Kumar Dey, Ruwad Naswan, Hasnaen Adil, Khondker Salman Sayeed, Haz Sameen Shahgir

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

ConceptSearch: Towards Efficient Program Search Using LLMs for Abstraction and Reasoning Corpus (ARC)

Kartik Singhal, Gautam Shroff

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；prompting(abstract)

Comments Pre-print of paper accepted at AAAI 2025

Alignment at Pre-training! Towards Native Alignment for Arabic LLMs

Juhao Liang, Zhenyang Cai, Jianqing Zhu, Huang Huang, Kewei Zong, Bang An, Mosen Alharthi, Juncai He, Lian Zhang, Haizhou Li, Benyou Wang, Jinchao Xu

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments Accepted to NeurIPS 2024 main conference. see https://github.com/FreedomIntelligence/AceGPT-v2

RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process

Peiran Wang, Xiaogeng Liu, Chaowei Xiao

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Theoretical Analysis of Byte-Pair Encoding

László Kozma, Johannes Voderholzer

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, Thomas Wolf

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Persistent Pre-Training Poisoning of LLMs

Yiming Zhang, Javier Rando, Ivan Evtimov, Jianfeng Chi, Eric Michael Smith, Nicholas Carlini, Florian Tramèr, Daphne Ippolito

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；post-training(abstract)；SFT(abstract)

Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed?

Tannon Kew, Florian Schottmann, Rico Sennrich

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；instruction tuning(abstract)；pretraining(abstract)

Comments Accepted at Findings of EMNLP 2024

Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale

Wenzhen Zheng, Wenbo Pan, Xu Xu, Libo Qin, Li Yue, Ming Zhou

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 8 pages. Accepted at EMNLP 2024

X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment

Dongjae Shin, Hyeonseok Lim, Inho Won, Changsu Choi, Minjun Kim, Seungwoo Song, Hangyeol Yoo, Sangmin Kim, Kyungtae Lim

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Journal ref https://aclanthology.org/2024.findings-naacl.158

Are Bigger Encoders Always Better in Vision Large Models?

Bozhou Li, Hao Liang, Zimo Meng, Wentao Zhang

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

On Machine Learning Approaches for Protein-Ligand Binding Affinity Prediction

Nikolai Schapin, Carles Navarro, Albert Bou, Gianni De Fabritiis

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Comments 20 pages, 14 figures, 1 table

Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer

Hele-Andra Kuulmets, Taido Purason, Agnes Luhtaru, Mark Fishel

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

Journal ref Findings of the Association for Computational Linguistics: NAACL 2024, pages 3309-3325

Efficient Continual Pre-training by Mitigating the Stability Gap

Yiduo Guo, Jie Fu, Huishuai Zhang, Dongyan Zhao, Yikang Shen

专题命中 预训练与数据 ：LLM(abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Maps

Dicong Qiu, Wenzong Ma, Zhenfu Pan, Hui Xiong, Junwei Liang

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；pretraining(abstract)；SFT(abstract)

Comments Open-vocabulary, Mobile Manipulation, Dynamic Environments, 3D Semantic Maps, Zero-shot, LLMs, VLMs, 18 pages, 2 figures

Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI

Wei-Bang Jiang, Li-Ming Zhao, Bao-Liang Lu

专题命中 预训练与数据 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；prompting(abstract)

Comments The Twelfth International Conference on Learning Representations

Journal ref The Twelfth International Conference on Learning Representations, 2024