语言大模型 / LLM - arXivDaily 专题

2104.14899 2021-05-03 cs.IR cs.AI 74%

Improving Conversational Recommendation System by Pretraining on Billions Scale of Knowledge Graph

Chi-Man Wong, Fan Feng, Wen Zhang, Chi-Man Vong, Hui Chen, Yichi Zhang, Peng He, Huan Chen, Kun Zhao, Huajun Chen

专题命中预训练与数据：pretraining(title)；分类 cs.AI

Comments Paper is accepted by ICDE2021 industry track

2104.10661 2021-04-22 cs.CL 74%

Towards Automated Psychotherapy via Language Modeling

Houjun Liu

专题命中预训练与数据：language model(title)；分类 cs.CL

Comments 10 pages, 5 figures

2104.01572 2021-04-06 cs.CL 74%

TransfoRNN: Capturing the Sequential Information in Self-Attention Representations for Language Modeling

Tze Yuang Chong, Xuyang Wang, Lin Yang, Junjie Wang

专题命中预训练与数据：language model(title)；分类 cs.CL

Comments INTERSPEECH 2021 (under reviewed)

2006.02049 2021-04-01 cs.CV cs.LG cs.NE 74%

FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining

Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, Joseph E. Gonzalez

专题命中预训练与数据：pretraining(title)；分类 cs.LG

2103.01988 2021-03-08 cs.CV cs.AI 74%

Self-supervised Pretraining of Visual Features in the Wild

Priya Goyal, Mathilde Caron, Benjamin Lefaudeux, Min Xu, Pengchao Wang, Vivek Pai, Mannat Singh, Vitaliy Liptchinsky, Ishan Misra, Armand Joulin, Piotr Bojanowski

专题命中预训练与数据：pretraining(title)；分类 cs.AI

2103.01032 2021-03-02 cs.CL cs.SD eess.AS q-bio.NC 74%

Inductive biases, pretraining and fine-tuning jointly account for brain responses to speech

Juliette Millet, Jean-Remi King

专题命中预训练与数据：pretraining(title)；分类 cs.CL

Comments 10 pages, 3 figures

2102.09681 2021-02-22 cs.CL cs.IR 74%

WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web

Robert Ormandi, Mohammad Saleh, Erin Winter, Vinay Rao

专题命中预训练与数据：pretraining(title)；分类 cs.CL

2010.12401 2020-10-26 cs.CL 74%

Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets

Gaurish Thakkar, Marcis Pinnis

专题命中预训练与数据：pretraining(title)；分类 cs.CL

2008.09144 2020-10-12 cs.CL 74%

PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data

Diedre Carmo, Marcos Piau, Israel Campiotti, Rodrigo Nogueira, Roberto Lotufo

专题命中预训练与数据：pretraining(title)；分类 cs.CL

2004.02016 2020-09-22 cs.CL 74%

A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining

Chenguang Zhu, Ruochen Xu, Michael Zeng, Xuedong Huang

专题命中预训练与数据：pretraining(title)；分类 cs.CL

Comments Accepted in Empirical Methods in Natural Language Processing (EMNLP), 2020

2008.03979 2020-08-12 cs.CL 74%

KR-BERT: A Small-Scale Korean-Specific Language Model

Sangah Lee, Hansol Jang, Yunmee Baik, Suzi Park, Hyopil Shin

专题命中预训练与数据：language model(title)；分类 cs.CL

Comments 7 pages

2006.01912 2020-06-04 cs.CL 74%

On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior

Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu, Peng Qian, Roger Levy

专题命中预训练与数据：language model(title)；分类 cs.CL

Comments To Appear at CogSci 2020

2005.14613 2020-06-01 cs.CL cs.IR 74%

Using Large Pretrained Language Models for Answering User Queries from Product Specifications

Kalyani Roy, Smit Shah, Nithish Pai, Jaidam Ramtej, Prajit Prashant Nadkarn, Jyotirmoy Banerjee, Pawan Goyal, Surender Kumar

专题命中预训练与数据：language model(title)；分类 cs.CL

Comments 5 pages

2003.13525 2020-03-31 cs.CV cs.LG 74%

Improving out-of-distribution generalization via multi-task self-supervised pretraining

Isabela Albuquerque, Nikhil Naik, Junnan Li, Nitish Keskar, Richard Socher

专题命中预训练与数据：pretraining(title)；分类 cs.LG

1912.10169 2019-12-24 cs.CL 74%

A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings

Niels van der Heijden, Samira Abnar, Ekaterina Shutova

专题命中预训练与数据：pretraining(title)；分类 cs.CL

Comments 7 pages, 6 figures

1909.00111 2019-09-04 cs.CL 74%

Quantity doesn't buy quality syntax with neural language models

Marten van Schijndel, Aaron Mueller, Tal Linzen

专题命中预训练与数据：language model(title)；分类 cs.CL

Comments Accepted for presentation at EMNLP-IJCNLP 2019

1908.02265 2019-08-07 cs.CV cs.CL 74%

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

Jiasen Lu, Dhruv Batra, Devi Parikh, Stefan Lee

专题命中预训练与数据：pretraining(title)；分类 cs.CL

Comments 11 pages, 5 figures

1811.02062 2018-11-07 cs.CL cs.SD eess.AS 74%

End-to-End Monaural Multi-speaker ASR System without Pretraining

Xuankai Chang, Yanmin Qian, Kai Yu, Shinji Watanabe

专题命中预训练与数据：pretraining(title)；分类 cs.CL

Comments submitted to ICASSP2019

1710.02076 2017-10-06 cs.CL 74%

On the Effective Use of Pretraining for Natural Language Inference

Ignacio Cases, Minh-Thang Luong, Christopher Potts

专题命中预训练与数据：pretraining(title)；分类 cs.CL

Comments This manuscript dates from late Winter 2016

1704.04565 2017-08-22 cs.CL 74%

Neural Paraphrase Identification of Questions with Noisy Pretraining

Gaurav Singh Tomar, Thyago Duque, Oscar Täckström, Jakob Uszkoreit, Dipanjan Das

专题命中预训练与数据：pretraining(title)；分类 cs.CL

1701.08702 2017-01-31 cs.CL 74%

Bangla Word Clustering Based on Tri-gram, 4-gram and 5-gram Language Model

Dipaloke Saha, Md Saddam Hossain, MD. Saiful Islam, Sabir Ismail

专题命中预训练与数据：language model(title)；分类 cs.CL

Comments 6 pages

2608.23391 2026-08-25 cs.CL cs.AI 新提交 73%

Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data

无域内训练数据的跨域多任务数据到文本生成

Yifei Song, Kun Efimov-Zhang, Claire Gardent

机构 * CNRS/LORIA（法国国家科学研究中心/洛林计算机科学与应用实验室）； Université de Lorraine（洛林大学）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本研究提出数据驱动知识蒸馏（DDKD）及保结构增强方法，在无域内训练数据的跨域多任务数据到文本生成任务中，17亿参数的DDKD性能优于零样本推理与微调，且构建了QUINTD-5数据集。

Comments Accepted by EMNLP Findings 2026

2608.22071 2026-08-25 cs.CL cs.AI 新提交 73%

Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction

Real-TurnTurk：用于话轮转换预测的多模态土耳其语语料库

Ahmet Tuğrul Bayrak, Fatma Nur Korkmaz, Bekir Berker Türker, Mustafa Sertaç Türkel, Alper Kaplan

机构 * Data Science and Innovation Ata Technology Platforms（阿塔技术平台数据科学与创新部）； Digital Operations and Data（数字运营与数据部）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本研究构建多模态土耳其语对话数据集，采用遗传算法优化规则，实现话轮转换预测，填补土耳其语相关自然对话语料库的空白。

Comments Accepted to INTCEC 2026. This is the author's pre-print version. The final authenticated version will be available through the conference proceedings

2608.21702 2026-08-25 cs.AI cs.CL 新提交 73%

From Association to Causation: Improving Retrieval Precision of Retrieval-Augmented Generation via Causal Relations and an Attention Mechanism

从关联到因果：通过因果关系与注意力机制提升检索增强生成的检索精度

Jing Liu, Yongxing Qi, Muchen Jiang, Chengnan Hu, Qingqing Peng, Haoming Wang, Yuqing Wang, Yang Yu, Xu Zhang, Ting Wu

机构 * Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）

专题命中预训练与数据：LLM(abstract,abstract_cn)；分类 cs.CL、cs.AI

AI总结该研究针对RAG检索阶段仅捕捉关联关系的问题，通过建模检索过程的因果结构，提出一种无需训练的注意力式重评分规则，在企业知识库和关键词堆砌语料库上显著提升了检索精度。

Comments 16 pages, 2 figures, 4 tables, 1 algorithm. Code available at https://github.com/Silk-Road/causal-rag-rerank

2608.12750 2026-08-25 cs.CL cs.AI cs.HC 版本更新 73%

PatientAct: Theory-Grounded Mental Health Client Simulation

PatientAct：基于理论的心理健康来访者模拟

Sahand Sabour, TszYam NG, Yaqian Chen, Guanqun Bi, Jialu Zhao, Minlie Huang

机构 * Tsinghua University（清华大学）； Beijing Normal University（北京师范大学）

专题命中预训练与数据：LLM(abstract,abstract_cn)；分类 cs.CL、cs.AI

AI总结 PatientAct是基于临床理论的来访者模拟框架，通过整合5Ps临床案例 formulation与带信任阈值的动态记忆层，生成高临床合理性的多样化来访者，在40种临床情境中较基线方法显著提升了抗拒质量与行为真实性。

Comments EMNLP 2026 Findings

2608.21252 2026-08-24 cs.CL cs.AI cs.DB cs.IR 新提交 73%

EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering

EnSI-RAG：面向长文档问答的实体结构索引增强检索生成框架

Xuanyu Meng, Jiashuo Sun, Jash Rajesh Parekh, Jiawei Han

专题命中预训练与数据：LLM(abstract,abstract_cn)；分类 cs.CL、cs.AI

AI总结该研究针对长文档问答的RAG方法缺陷，提出EnSI-RAG框架，构建以实体为中心的索引，在Loong和Oolong数据集上平均准确率达78.24，较基准提升6.62个百分点，验证了其有效性。

Comments 21 pages, preprint

2606.13629 2026-08-24 stat.ME cs.AI cs.LG stat.ML 版本更新 73%

Valid Inference with Synthetic Data via Task Exchangeability

通过任务可交换性实现基于合成数据的有效推断

Lezhi Tan, Tijana Zrnic

机构 * Department of Management Science & Engineering（管理科学与工程系）； Department of Statistics（统计学系）

专题命中预训练与数据：LLM(abstract,abstract_cn)；分类 cs.AI、cs.LG

AI总结提出任务可交换性条件，确保在科学研究中使用合成数据进行统计推断的有效性，并给出在民意调查和AI评估中的应用。

2608.18821 2026-08-20 cs.CL cs.AI 新提交 73%

Identifying Implicit Premises for Logical Reconstruction of Argument Graphs

识别论证图逻辑重构的隐含前提

Xuyao Feng, Anthony Hunter

机构 * University College London（伦敦大学学院）

专题命中预训练与数据：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AI总结本文针对论证图逻辑重构中省略式论证的隐含前提问题，提出神经符号管道方法，结合大型语言模型生成隐含前提并评估，用于证明陈述间的逻辑关系。

Comments Accepted at the 11th International Conference on Computational Models of Argument (COMMA 2026)

2608.18062 2026-08-19 cs.CL cs.LG 新提交 73%

TokEval: A Tokenizer Evaluation Suite

TokEval：一个分词器评估套件

Clara Meister

机构 * EPFL（洛桑联邦理工学院）

专题命中预训练与数据：language model(abstract)；pretraining(abstract)；分类 cs.CL、cs.LG

AI总结本研究推出TokEval框架，通过信息论、结构敏感等指标评估分词器，经实验验证其可预测下游模型性能，助力更原则性的分词器评估。

Comments Published as a conference paper at COLM 2026; Library hosted at https://github.com/cimeister/tokenizer-intrinsic-evals

2608.16185 2026-08-19 cs.CL cs.AI 版本更新 73%

LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents

LENS：基于动态原始文档的潜在证据探索的上下文内搜索

Xingjun Wang, Gongsheng Li, Qi Fan, Yunlin Mao, Luyan Su, Yingda Chen

专题命中预训练与数据：LLM(abstract,abstract_cn)；分类 cs.CL、cs.AI

AI总结针对动态原始文档的上下文内搜索问题，提出无索引框架LENS，通过迭代选择候选证据并更新置信度实现预算约束下的证据定位，在多组实验中其证据召回率和答案依据表现优于基线方法。

1. 预训练与数据 12429 篇

Improving Conversational Recommendation System by Pretraining on Billions Scale of Knowledge Graph

Towards Automated Psychotherapy via Language Modeling

TransfoRNN: Capturing the Sequential Information in Self-Attention Representations for Language Modeling

FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining

Self-supervised Pretraining of Visual Features in the Wild

Inductive biases, pretraining and fine-tuning jointly account for brain responses to speech

WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web

Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets

PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data

A Hierarchical Network for Abstractive Meeting Summarization with Cross-Domain Pretraining

KR-BERT: A Small-Scale Korean-Specific Language Model

On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior

Using Large Pretrained Language Models for Answering User Queries from Product Specifications

Improving out-of-distribution generalization via multi-task self-supervised pretraining

A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings

Quantity doesn't buy quality syntax with neural language models

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

End-to-End Monaural Multi-speaker ASR System without Pretraining

On the Effective Use of Pretraining for Natural Language Inference

Neural Paraphrase Identification of Questions with Noisy Pretraining

Bangla Word Clustering Based on Tri-gram, 4-gram and 5-gram Language Model

Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data

Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction

From Association to Causation: Improving Retrieval Precision of Retrieval-Augmented Generation via Causal Relations and an Attention Mechanism

PatientAct: Theory-Grounded Mental Health Client Simulation

EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering

Valid Inference with Synthetic Data via Task Exchangeability

Identifying Implicit Premises for Logical Reconstruction of Argument Graphs

TokEval: A Tokenizer Evaluation Suite

LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents