语言大模型 / LLM - arXivDaily 专题

2404.02650 2024-04-04 cs.LG cs.AI cs.CL 89%

Towards detecting unanticipated bias in Large Language Models

Anna Kruspe

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2403.13073 2024-03-21 cs.CY 89%

A Canary in the AI Coal Mine: American Jews May Be Disproportionately Harmed by Intellectual Property Dispossession in Large Language Model Training

Heila Precel, Allison McDonald, Brent Hecht, Nicholas Vincent

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments Preprint, to appear in CHI 2024 proceedings

2403.00758 2024-03-21 cs.CL cs.AI cs.LG 89%

Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training

Qingyan Guo, Rui Wang, Junliang Guo, Xu Tan, Jiang Bian, Yujiu Yang

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2403.07874 2024-03-13 cs.CV 89%

Beyond Text: Frozen Large Language Models in Visual Signal Comprehension

Lei Zhu, Fangyun Wei, Yanye Lu

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments Accepted by CVPR 2024

2402.15313 2024-02-27 cs.CL cs.AI cs.LG 89%

ArabianGPT: Native Arabic GPT-based Large Language Model

Anis Koubaa, Adel Ammar, Lahouari Ghouti, Omar Najar, Serry Sibaee

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2402.03471 2024-02-07 cs.LG cs.AI cs.CL cs.IT math.IT 89%

The Information of Large Language Model Geometry

Zhiquan Tan, Chenghai Li, Weiran Huang

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2310.15140 2023-12-15 cs.CR cs.AI cs.CL cs.LG 89%

AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models

Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, Tong Sun

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Version 2 updates: Added comparison of three more evaluation methods and their reliability check using human labeling. Added results for jailbreaking Llama2 (individual behavior) and included complexity and hyperparameter analysis. Revised objectives for prompt leaking. Other minor changes made

2311.13778 2023-11-27 cond-mat.mtrl-sci 89%

Accurate Prediction of Experimental Band Gaps from Large Language Model-Based Data Extraction

Samuel J. Yang, Shutong Li, Subhashini Venugopalan, Vahe Tshitoyan, Muratahan Aykol, Amil Merchant, Ekin Dogus Cubuk, Gowoon Cheon

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2303.03012 2023-11-01 cs.SE 89%

On Extracting Specialized Code Abilities from Large Language Models: A Feasibility Study

Zongjie Li, Chaozheng Wang, Pingchuan Ma, Chaowei Liu, Shuai Wang, Daoyuan Wu, Cuiyun Gao, Yang Liu

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

Comments 13 pages

2310.13014 2023-10-23 cs.CY cs.AI cs.CL cs.LG 89%

Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament

Philipp Schoenegger, Peter S. Park

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 13 pages, six visualizations (four figures, two tables)

2310.08256 2023-10-13 cs.CL cs.AI cs.LG 89%

Impact of Co-occurrence on Factual Knowledge of Large Language Models

Cheongwoong Kang, Jaesik Choi

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2023 Findings

2306.11732 2023-06-21 cs.CV 89%

Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models

Junting Pan, Ziyi Lin, Yuying Ge, Xiatian Zhu, Renrui Zhang, Yi Wang, Yu Qiao, Hongsheng Li

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；LLM(abstract)

2212.09282 2023-06-06 cs.CL cs.AI cs.LG 89%

APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning

Soumya Sanyal, Yichong Xu, Shuohang Wang, Ziyi Yang, Reid Pryzant, Wenhao Yu, Chenguang Zhu, Xiang Ren

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2023, code available at https://github.com/INK-USC/APOLLO

2305.12223 2023-05-24 cs.CV 89%

What Makes for Good Visual Tokenizers for Large Language Models?

Guangzhi Wang, Yixiao Ge, Xiaohan Ding, Mohan Kankanhalli, Ying Shan

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；pretraining(abstract)

Comments 15 pages, 3 figures. Project released at: https://github.com/TencentARC/GVT

2305.12268 2023-05-23 cs.CL cs.AI cs.LG 89%

Patton: Language Model Pretraining on Text-Rich Networks

Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, Jiawei Han

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2023. (Code: https://github.com/PeterGriffinJin/Patton)

2205.10475 2023-03-07 cs.CL cs.AI cs.LG 89%

DeepStruct: Pretraining of Language Models for Structure Prediction

Chenguang Wang, Xiao Liu, Zui Chen, Haoyun Hong, Jie Tang, Dawn Song

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2022

2210.04782 2023-02-14 cs.CL cs.AI cs.LG 89%

Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining

Asa Cooper Stickland, Sailik Sengupta, Jason Krone, Saab Mansour, He He

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted and to be presented at the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2023

2205.08184 2022-05-18 cs.CL cs.AI cs.LG 89%

SKILL: Structured Knowledge Infusion for Large Language Models

Fedor Moiseev, Zhe Dong, Enrique Alfonseca, Martin Jaggi

专题命中预训练与数据：large language model(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NAACL 2022

2103.10360 2022-03-18 cs.CL cs.AI cs.LG 89%

GLM: General Language Model Pretraining with Autoregressive Blank Infilling

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments to be published in ACL 2022. 16 pages, 4 figures

2103.10685 2021-11-10 cs.CL cs.AI cs.LG 89%

Controllable Generation from Pre-trained Language Models via Inverse Prompting

Xu Zou, Da Yin, Qingyang Zhong, Ming Ding, Hongxia Yang, Zhilin Yang, Jie Tang

专题命中预训练与数据：language model(title,abstract)；prompting(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Slightly different from the KDD version

2109.01819 2021-09-07 cs.CL cs.AI cs.LG 89%

Frustratingly Simple Pretraining Alternatives to Masked Language Modeling

Atsuki Yamaguchi, George Chrysostomou, Katerina Margatina, Nikolaos Aletras

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at EMNLP 2021

2103.07040 2021-03-15 cs.CL cs.AI cs.LG 89%

Bilingual Dictionary-based Language Model Pretraining for Neural Machine Translation

Yusen Lin, Jiayong Lin, Shuaicheng Zhang, Haoying Dai

专题命中预训练与数据：language model(title,abstract)；pretraining(title,abstract)；分类 cs.CL、cs.AI、cs.LG

2606.09912 2026-06-10 cs.LG cs.AI 新提交 89%

Mix, Don't Pick: Why Synthetic Corpus Composition Matters for Time Series Foundation Model Pretraining

混合而非挑选：为什么合成语料组合对时间序列基础模型预训练至关重要

Aaryan Nagpal, Debdeep Sanyal, Murari Mandal, Dhruv Kumar, Saurabh Deshpande

机构 * Birla AI Labs（巴尔拉人工智能实验室）

专题命中预训练与数据：foundation model(title,abstract)；pretraining(title,abstract)；分类 cs.AI、cs.LG

AI总结针对时间序列基础模型预训练中合成数据生成器选择困难的问题，提出简单等权混合所有生成器的方法，匹配或超越最优单个生成器，并与真实数据结合获得最强预训练语料。

Comments Accepted at the ICML 2026 Workshop on Foundation Models for Structured Data (FMSD), Seoul, South Korea

2605.12241 2026-05-13 eess.SP cs.AI cs.LG 89%

Pretraining Strategies and Scaling for ECG Foundation Models: A Systematic Study

ECG基础模型的预训练策略与扩展：一项系统研究

M A Al-Masud, Nils Strodthoff

机构 * AI4Health Division（AI4Health部门）

专题命中预训练与数据：pretraining(title,abstract)；foundation model(title,abstract)；分类 cs.AI、cs.LG

AI总结本文系统评估了ECG基础模型的预训练方法，探讨了不同对比和非对比自监督学习目标在不同预训练数据集规模下的表现，发现对比预测编码在跨临床任务的迁移性能上表现最佳。

Comments 59 pages, 16 figures, 59 Tables. Code available at https://anonymous.4open.science/r/ecg-pretraining-strategies-4DE3

2108.04106 2022-03-16 cs.CL cs.AI 89%

Noisy Channel Language Model Prompting for Few-Shot Text Classification

Sewon Min, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer

专题命中预训练与数据：prompting(title,abstract)；language model(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 6 figures. Published as a conference paper at ACL 2022 (long). Code available at https://github.com/shmsw25/Channel-LM-Prompting

2608.14591 2026-08-18 eess.SP cs.IT cs.LG math.IT 新提交 89%

6G Native AI and Channel Foundation Models

6G原生AI与信道基础模型

Shugong Xu, Jun Jiang, Yuan Gao

专题命中预训练与数据：foundation model(title,abstract)；large language model(abstract)；language model(abstract)；pretraining(abstract)

AI总结本文从系统设计视角界定6G原生AI的内涵，提出信道基础模型（CFMs）作为6G原生AI的技术范式，阐明其类型与优势，验证其在标注有限时可提升定位与波束预测性能。

Comments Awesome GitHub: https://github.com/GREAT-ISAC/Awesome-Channel-Foundation-Models

2608.14229 2026-08-17 cs.CL 新提交 89%

The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning

越受欢迎，越难遗忘：用于大语言模型遗忘的自适应受欢迎度

Anna Borisiuk, Andrey Savchenko, Alexander Panchenko, Elena Tutubalina

机构 * AIRI ； Sber AI Lab（Sber AI实验室）； Skoltech（斯科尔科沃科学技术研究院）； ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息学研究所可信人工智能研究中心）

专题命中预训练与数据：LLM(title,summary_cn)；pretraining(abstract)；分类 cs.CL

AI总结针对现有LLM遗忘方法未区分训练数据频率的问题，提出AdaPop方法，经实验其在转述查询和对抗性重构下均能显著减少遗忘内容泄露。

2608.10690 2026-08-12 cs.CL 新提交 89%

Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora?

已发布的大语言模型词汇表能否支持隐藏语料库的词元级估计？

Qingjie Zhang, Xingzhang Ren, Zixuan Chen, Jinfeng Li, YueFeng Chen, Yitong Yang, Hui Xue, Dayiheng Liu, Han Qiu

机构 * Tsinghua University（清华大学）； Alibaba Group（阿里巴巴集团）

专题命中预训练与数据：LLM(title,summary_cn)；pretraining(abstract)；分类 cs.CL

AI总结该研究针对隐藏语料库的词元级估计问题，提出分位数引导密度估计（QGDE）方法，利用已发布的LLM分词器词汇表实现了低误差的细粒度语料估计，为相关任务提供了新的有效信号来源。

2607.13416 2026-07-16 cs.LG 新提交 89%

EXPLORE: Exploration with Guided Search for Analog Topology Generation using Language Models

EXPLORE：使用语言模型进行引导搜索以生成模拟拓扑结构

Guanglei Zhou, Chen-Chia Chang, Yikang Shen, Jonathan Ku, Isaac Jacobson, Jingyu Pan, Yiran Chen, Xin Zhang

机构 * Duke University（杜克大学）； MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）； IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）

专题命中预训练与数据：LLM(summary_cn,abstract)；language model(title,abstract)；分类 cs.LG

AI总结本文针对自动化模拟电路拓扑设计难题，提出EXPLORE框架，集成模拟器引导蒙特卡罗树搜索与基于变压器的解码，利用语言模型先验优化搜索，在6组件基准测试中显著提升成功率、降低均方误差，推动LLM驱动设计自动化。

Comments MLCAD 26' accepted

2607.01733 2026-07-03 cs.CL eess.AS 新提交 89%

Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving

重新思考语音-LLM集成用于ASR：通过交错实现有效的联合语音-文本训练

Ruchao Fan, Yiming Wang, Rui Zhao, Liliang Ren, Keqi Deng, Xiaoyang Chen, Ali Zare, Bo Ren, Yuxuan Hu, Junkun Chen, Yan Huang, Yelong Shen, Jinyu Li

机构 * Microsoft（微软）

专题命中预训练与数据：LLM(title,title_cn)；pretraining(abstract)；分类 cs.CL

AI总结提出JSTIP，一种面向ASR的预训练策略，通过构建词级和段级交错语音-文本序列，在38k小时数据上相比基线提升了实体识别准确率，并缩小了模态差距。

1. 预训练与数据 12403 篇

Towards detecting unanticipated bias in Large Language Models

A Canary in the AI Coal Mine: American Jews May Be Disproportionately Harmed by Intellectual Property Dispossession in Large Language Model Training

Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training

Beyond Text: Frozen Large Language Models in Visual Signal Comprehension

ArabianGPT: Native Arabic GPT-based Large Language Model

The Information of Large Language Model Geometry

AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models

Accurate Prediction of Experimental Band Gaps from Large Language Model-Based Data Extraction

On Extracting Specialized Code Abilities from Large Language Models: A Feasibility Study

Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament

Impact of Co-occurrence on Factual Knowledge of Large Language Models

Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models

APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning

What Makes for Good Visual Tokenizers for Large Language Models?

Patton: Language Model Pretraining on Text-Rich Networks

DeepStruct: Pretraining of Language Models for Structure Prediction

Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining

SKILL: Structured Knowledge Infusion for Large Language Models

GLM: General Language Model Pretraining with Autoregressive Blank Infilling

Controllable Generation from Pre-trained Language Models via Inverse Prompting

Frustratingly Simple Pretraining Alternatives to Masked Language Modeling

Bilingual Dictionary-based Language Model Pretraining for Neural Machine Translation

Mix, Don't Pick: Why Synthetic Corpus Composition Matters for Time Series Foundation Model Pretraining

Pretraining Strategies and Scaling for ECG Foundation Models: A Systematic Study

Noisy Channel Language Model Prompting for Few-Shot Text Classification

6G Native AI and Channel Foundation Models

The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning

Can Released LLM Vocabularies Support Token-Level Estimation of Hidden Corpora?

EXPLORE: Exploration with Guided Search for Analog Topology Generation using Language Models

Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving