语言大模型 / LLM - arXivDaily 专题

2404.08679 2025-03-07 cs.CL cs.AI cs.LG stat.ML 90%

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector

Andi Zhang, Tim Z. Xiao, Weiyang Liu, Robert Bamler, Damon Wischik

机构 * University of Manchester（曼彻斯特大学）； University of Tübingen（蒂宾根大学）； University of Cambridge（剑桥大学）； Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2404.13885 2025-01-17 cs.CY cs.AI cs.CL cs.LG 90%

Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals

Qingyang Wu, Ying Xu, Tingsong Xiao, Yunze Xiao, Yitong Li, Tianyang Wang, Yichi Zhang, Shanghai Zhong, Yuwei Zhang, Wei Lu, Yifan Yang

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2412.15285 2024-12-23 cs.CL cs.AI cs.LG 90%

Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining

Steven Feng, Shrimai Prabhumoye, Kezhi Kong, Dan Su, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro

机构 * NVIDIA（英伟达）； Stanford University（斯坦福大学）； Boston University（波士顿大学）

专题命中预训练与数据：pretraining(title,abstract)；LLM(title)；large language model(abstract)；language model(abstract)

2412.01014 2024-12-03 cs.LG cs.AI cs.CL 90%

Detecting Memorization in Large Language Models

Eduardo Slonski

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2411.07990 2024-11-13 cs.CL cs.AI cs.LG 90%

Derivational Morphology Reveals Analogical Generalization in Large Language Models

Valentin Hofmann, Leonie Weissweiler, David Mortensen, Hinrich Schütze, Janet Pierrehumbert

机构 * Allen Institute for AI（艾伦人工智能研究所）； LMU Munich（慕尼黑大学）； Carnegie Mellon University（卡内基梅隆大学）； University of Oxford（牛津大学）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2410.05269 2024-10-08 cs.CL cs.AI cs.LG 90%

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models

Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2024 Main Conference. Project website: https://feiwang96.github.io/DataAdvisor/

2403.08763 2024-09-05 cs.LG cs.AI cs.CL 90%

Simple and Scalable Strategies to Continually Pre-train Large Language Models

Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L. Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, Irina Rish

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2405.19327 2024-07-11 cs.CL cs.AI cs.LG 90%

MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series

Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Esther Cheng, Jie Liu, Qunshu Lin, Raven Yuan, Tuney Zheng, Wei Pang, Xinrun Du, Yiming Liang, Yinghao Ma, Yizhi Li, Ziyang Ma, Bill Lin, Emmanouil Benetos, Huan Yang, Junting Zhou, Kaijing Ma, Minghao Liu, Morry Niu, Noah Wang, Quehry Que, Ruibo Liu, Sine Liu, Shawn Guo, Soren Gao, Wangchunshu Zhou, Xinyue Zhang, Yizhi Zhou, Yubo Wang, Yuelin Bai, Yuhan Zhang, Yuxiang Zhang, Zenith Wang, Zhenzhu Yang, Zijian Zhao, Jiajun Zhang, Wanli Ouyang, Wenhao Huang, Wenhu Chen

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments https://map-neo.github.io/

2309.08631 2024-06-06 cs.CL cs.AI cs.CY cs.HC cs.LG cs.SI 90%

Large Language Models Can Infer Psychological Dispositions of Social Media Users

Heinrich Peters, Sandra Matz

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2405.10140 2024-05-17 cs.CV 90%

Libra: Building Decoupled Vision System on Large Language Models

Yifan Xu, Xiaoshan Yang, Yaguang Song, Changsheng Xu

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；foundation model(abstract)

Comments ICML2024

2404.12299 2024-04-19 cs.CL cs.AI cs.LG cs.SD eess.AS 90%

Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair

Yusuke Sakai, Mana Makinae, Hidetaka Kamigaito, Taro Watanabe

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages, 9 figures

2311.16926 2024-04-04 cs.CV 90%

LLaFS: When Large Language Models Meet Few-Shot Segmentation

Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；pretraining(abstract)

Comments Accepted to CVPR2024

2403.19031 2024-03-29 cs.CL cs.AI cs.LG 90%

Evaluating Large Language Models for Health-Related Text Classification Tasks with Public Social Media Data

Yuting Guo, Anthony Ovadje, Mohammed Ali Al-Garadi, Abeed Sarker

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2403.14683 2024-03-25 cs.CY cs.AI cs.CL cs.LG 90%

A Moral Imperative: The Need for Continual Superalignment of Large Language Models

Gokul Puthumanaillam, Manav Vora, Pranay Thangeda, Melkior Ornik

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2309.04669 2024-03-25 cs.CV 90%

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Yang Jin, Kun Xu, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Quzhe Huang, Bin Chen, Chenyi Lei, An Liu, Chengru Song, Xiaoqiang Lei, Di Zhang, Wenwu Ou, Kun Gai, Yadong Mu

专题命中预训练与数据：LLM(title,abstract)；pretraining(title)；large language model(abstract)；language model(abstract)

Comments ICLR 2024

2403.10557 2024-03-19 cs.LG cs.AI cs.CL 90%

Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models

Kang Gu, Md Rafi Ur Rashid, Najrin Sultana, Shagufta Mehnaz

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2403.00198 2024-03-04 cs.CL cs.AI cs.CY cs.LG 90%

AXOLOTL: Fairness through Assisted Self-Debiasing of Large Language Model Outputs

Sana Ebrahimi, Kaiwen Chen, Abolfazl Asudeh, Gautam Das, Nick Koudas

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2402.01981 2024-02-06 cs.CL cs.AI cs.CY cs.LG 90%

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes

Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Tong Yu, Hanieh Deilamsalehy, Ruiyi Zhang, Sungchul Kim, Franck Dernoncourt

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2401.13086 2024-01-25 cs.CL cs.AI cs.LG 90%

Towards Trustable Language Models: Investigating Information Quality of Large Language Models

Rick Rejeleene, Xiaowei Xu, John Talburt

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 31 pages

2312.14226 2023-12-25 cs.CL cs.AI cs.LG stat.ML 90%

Deep de Finetti: Recovering Topic Distributions from Large Language Models

Liyi Zhang, R. Thomas McCoy, Theodore R. Sumers, Jian-Qiao Zhu, Thomas L. Griffiths

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 13 pages, 4 figures

2310.17784 2023-11-15 cs.CL cs.AI cs.LG 90%

Data-Centric Financial Large Language Models

Zhixuan Chu, Huaiyu Guo, Xinyuan Zhou, Yijia Wang, Fei Yu, Hong Chen, Wanqing Xu, Xin Lu, Qing Cui, Longfei Li, Jun Zhou, Sheng Li

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2310.15389 2023-10-25 cs.CL cs.AI cs.LG 90%

Irreducible Curriculum for Language Model Pretraining

Simin Fan, Martin Jaggi

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI、cs.LG

2306.15895 2023-10-19 cs.CL cs.AI cs.LG 90%

Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias

Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, Chao Zhang

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NeurIPS 2023 (Datasets and Benchmarks Track)

Journal ref NeurIPS 2023

2309.03748 2023-09-08 cs.CL cs.AI cs.LG stat.ML 90%

Enhancing Pipeline-Based Conversational Agents with Large Language Models

Mina Foosherian, Hendrik Purwins, Purna Rathnayake, Touhidul Alam, Rui Teimao, Klaus-Dieter Thoben

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

2308.09890 2023-08-25 cs.LG cs.AI cs.CL cs.PL 90%

Inductive-bias Learning: Generating Code Models with Large Language Model

Toma Tanaka, Naofumi Emoto, Tsukasa Yumibayashi

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 17 pages, 8 figures

2308.12284 2023-08-24 cs.CL cs.AI cs.LG 90%

D4: Improving LLM Pretraining via Document De-Duplication and Diversification

Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari S. Morcos

专题命中预训练与数据：LLM(title,abstract)；pretraining(title)；large language model(abstract)；language model(abstract)

2601.21725 2026-05-29 cs.CL cs.LG 90%

Procedural Pretraining: Warming Up Language Models with Abstract Data

程序化预训练：用抽象数据预热语言模型

Liangze Jiang, Zachary Shinnick, Anton van den Hengel, Hemanth Saratchandran, Damien Teney

机构 * EPFL（苏黎世联邦理工学院）； Idiap Research Institute（伊迪普研究机构）； AIML, Adelaide University（人工智能实验室，阿德莱德大学）

专题命中预训练与数据：pretraining(title,abstract)；language model(title,abstract)；LLM(abstract_cn)；分类 cs.CL、cs.LG

AI总结提出程序化预训练方法，通过在抽象结构化数据（如形式语言生成的程序数据）上预训练语言模型，显著提升其推理能力并加速后续语义知识学习，实验表明仅需0.1-0.3%的程序数据即可超越标准预训练。

Comments ICML 2026. Project page: https://zlshinnick.github.io/procedural-pretraining-page/

2508.14062 2025-08-21 cs.CL cs.AI 90%

Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models

Badrinath Ramakrishnan, Akshaya Balaji

机构 * Badrinath Ramakrishnan Akshaya Balaji（独立研究者）

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI

Comments 14 pages, 2 figures. Code and experimental framework available at https://github.com/akshayaaa10/llm-privacy-research

2505.16276 2025-05-23 cs.AI cs.CL 90%

How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance

Desiree Heim, Lars-Peter Meyer, Markus Schröder, Johannes Frey, Andreas Dengel

专题命中预训练与数据：language model(title,abstract)；large language model(title,abstract)；LLM(abstract)；分类 cs.CL、cs.AI

Comments Peer reviewed and to appear in the ESWC 2025 Workshops and Tutorials Joint Proceedings (Workshop on Evaluation of Language Models in Knowledge Engineering [ELMKE])

2405.06676 2024-05-14 cs.CL cs.AI cs.AR 90%

EDA Corpus: A Large Language Model Dataset for Enhanced Interaction with OpenROAD

Bing-Yue Wu, Utsav Sharma, Sai Rahul Dhanvi Kankipati, Ajay Yadav, Bintu Kappil George, Sai Ritish Guntupalli, Austin Rovinski, Vidya A. Chhabria

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract,comments)；分类 cs.CL、cs.AI

Comments Under review at Workshop on LLM-Aided Design (LAD'24)

1. 预训练与数据 12486 篇

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector

Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals

Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining

Detecting Memorization in Large Language Models

Derivational Morphology Reveals Analogical Generalization in Large Language Models

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models

Simple and Scalable Strategies to Continually Pre-train Large Language Models

MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series

Large Language Models Can Infer Psychological Dispositions of Social Media Users

Libra: Building Decoupled Vision System on Large Language Models

Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair

LLaFS: When Large Language Models Meet Few-Shot Segmentation

Evaluating Large Language Models for Health-Related Text Classification Tasks with Public Social Media Data

A Moral Imperative: The Need for Continual Superalignment of Large Language Models

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models

AXOLOTL: Fairness through Assisted Self-Debiasing of Large Language Model Outputs

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes

Towards Trustable Language Models: Investigating Information Quality of Large Language Models

Deep de Finetti: Recovering Topic Distributions from Large Language Models

Data-Centric Financial Large Language Models

Irreducible Curriculum for Language Model Pretraining

Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias

Enhancing Pipeline-Based Conversational Agents with Large Language Models

Inductive-bias Learning: Generating Code Models with Large Language Model

D4: Improving LLM Pretraining via Document De-Duplication and Diversification

Procedural Pretraining: Warming Up Language Models with Abstract Data

Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models

How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance

EDA Corpus: A Large Language Model Dataset for Enhanced Interaction with OpenROAD