语言大模型 / LLM - arXivDaily 专题

2311.17280 2023-12-27 cs.CL cs.CV 79%

Does VLN Pretraining Work with Nonsensical or Irrelevant Instructions?

Wang Zhu, Ishika Singh, Yuan Huang, Robin Jia, Jesse Thomason

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by O-DRUM @ CVPR 2023

2312.12764 2023-12-21 eess.AS cs.CL cs.SD 79%

Lattice Rescoring Based on Large Ensemble of Complementary Neural Language Models

Atsunori Ogawa, Naohiro Tawara, Marc Delcroix, Shoko Araki

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ICASSP 2022

2312.11927 2023-12-20 cs.LG cs.SI stat.ME 79%

Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery

Pengwei Yan, Kaisong Song, Zhuoren Jiang, Yangyang Kang, Tianqianjin Lin, Changlong Sun, Xiaozhong Liu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 14 pages, 6 figures, accepted by AAAI'24

2312.00874 2023-12-05 cs.CL 79%

Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining

Jingcong Liang, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments to be published in EMNLP 2023

2306.07848 2023-12-05 cs.CL cs.MM cs.SD eess.AS 79%

GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition

Yu Pan, Yanni Hu, Yuguang Yang, Wen Fei, Jixun Yao, Heng Lu, Lei Ma, Jianjun Zhao

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 5 pages

2312.00304 2023-12-04 cs.LG cs.CV 79%

Developmental Pretraining (DPT) for Image Classification Networks

Niranjan Rajesh, Debayan Gupta

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 7 pages, 4 figures

2312.00076 2023-12-04 cs.LG cs.CY cs.SI 79%

Towards A Foundation Model For Trajectory Intelligence

Alameen Najjar

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Accepted to the 2023 IEEE International Conference on Data Mining Workshops (ICDMW)

2311.18761 2023-12-01 cs.CL 79%

Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?

Aryaman Chobey, Oliver Smith, Anzi Wang, Grusha Prasad

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments To appear in the proceedings of BabyLM shared task CoNLL 2023

2305.14993 2023-12-01 cs.CL 79%

Controlling Pre-trained Language Models for Grade-Specific Text Simplification

Sweta Agrawal, Marine Carpuat

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP 2023

2311.16261 2023-11-29 cs.CV cs.AI 79%

RelVAE: Generative Pretraining for few-shot Visual Relationship Detection

Sotiris Karapiperis, Markos Diomataris, Vassilis Pitsikalis

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2311.11564 2023-11-21 cs.CL 79%

KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model

Lei Geng, Xu Yan, Ziqiang Cao, Juntao Li, Wenjie Li, Sujian Li, Xinjie Zhou, Yang Yang, Jun Zhang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2311.11477 2023-11-21 cs.CV cs.CL 79%

What's left can't be right -- The remaining positional incompetence of contrastive vision-language models

Nils Hoehing, Ellen Rushe, Anthony Ventresque

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2311.11368 2023-11-21 cs.LG cs.SI stat.ML 79%

Self-Supervised Pretraining for Heterogeneous Hypergraph Neural Networks

Abdalgader Abubaker, Takanori Maehara, Madhav Nimishakavi, Vassilis Plachouras

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2311.09205 2023-11-16 cs.CL 79%

When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages

Tyler A. Chang, Catherine Arnett, Zhuowen Tu, Benjamin K. Bergen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2311.04109 2023-11-08 cs.LG cs.CR 79%

Do Language Models Learn Semantics of Code? A Case Study in Vulnerability Detection

Benjamin Steenhoek, Md Mahbubur Rahman, Shaila Sharmin, Wei Le

专题命中预训练与数据：language model(title,abstract)；分类 cs.LG

2305.12908 2023-11-08 cs.CL 79%

Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training

Miriam Anschütz, Joshua Oehms, Thomas Wimmer, Bartłomiej Jezierski, Georg Groh

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL Findings 2023

2310.18343 2023-11-07 cs.CL 79%

PHD: Pixel-Based Language Modeling of Historical Documents

Nadav Borenstein, Phillip Rust, Desmond Elliott, Isabelle Augenstein

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to the main conference of EMNLP 2023

2311.00913 2023-11-03 cs.CL 79%

Self-Influence Guided Data Reweighting for Language Model Pre-training

Megh Thakkar, Tolga Bolukbasi, Sriram Ganapathy, Shikhar Vashishth, Sarath Chandar, Partha Talukdar

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2023

2311.00658 2023-11-02 cs.CL 79%

Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew

Eylon Gueta, Omer Goldman, Reut Tsarfaty

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2310.19089 2023-10-31 cs.CL 79%

Pushdown Layers: Encoding Recursive Structure in Transformer Language Models

Shikhar Murty, Pratyusha Sharma, Jacob Andreas, Christopher D. Manning

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at EMNLP 2023 (Long Papers)

2310.11166 2023-10-31 cs.CL 79%

ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing

Quoc-Nam Nguyen, Thang Chau Phan, Duc-Vu Nguyen, Kiet Van Nguyen

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at EMNLP'2023 Main Conference

2310.03274 2023-10-31 cs.LG 79%

Fragment-based Pretraining and Finetuning on Molecular Graphs

Kha-Dinh Luong, Ambuj Singh

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 18 pages, 4 figures, published in NeurIPS 2023

2305.16985 2023-10-26 cs.LG 79%

Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation

David Brandfonbrener, Ofir Nachum, Joan Bruna

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2306.14846 2023-10-25 cs.RO cs.CV cs.LG 79%

ViNT: A Foundation Model for Visual Navigation

Dhruv Shah, Ajay Sridhar, Nitish Dashora, Kyle Stachowicz, Kevin Black, Noriaki Hirose, Sergey Levine

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

Comments Accepted for oral presentation at CoRL 2023

2310.14840 2023-10-24 cs.CL 79%

Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution

Jaap Jumelet, Willem Zuidema

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EMNLP Findings 2023

2310.10567 2023-10-24 cs.CL 79%

RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling

Jingcheng Deng, Liang Pang, Huawei Shen, Xueqi Cheng

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to the Findings of EMNLP 2023

2310.14050 2023-10-24 cs.CL 79%

Code-Switching with Word Senses for Pretraining in Neural Machine Translation

Vivek Iyer, Edoardo Barba, Alexandra Birch, Jeff Z. Pan, Roberto Navigli

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EMNLP (Findings) 2023 Long Paper

2304.11389 2023-10-24 cs.CL 79%

Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens

Byung-Doh Oh, William Schuler

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Findings of the Association for Computational Linguistics: EMNLP 2023

2310.13286 2023-10-23 cs.IR cs.AI 79%

Unified Pretraining for Recommendation via Task Hypergraphs

Mingdai Yang, Zhiwei Liu, Liangwei Yang, Xiaolong Liu, Chen Wang, Hao Peng, Philip S. Yu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by WSDM 2024

2310.07871 2023-10-23 cs.AI 79%

Hierarchical Pretraining on Multimodal Electronic Health Records

Xiaochen Wang, Junyu Luo, Jiaqi Wang, Ziyi Yin, Suhan Cui, Yuan Zhong, Yaqing Wang, Fenglong Ma

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by EMNLP 2023

1. 预训练与数据 12429 篇

Does VLN Pretraining Work with Nonsensical or Irrelevant Instructions?

Lattice Rescoring Based on Large Ensemble of Complementary Neural Language Models

Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery

Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining

GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition

Developmental Pretraining (DPT) for Image Classification Networks

Towards A Foundation Model For Trajectory Intelligence

Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?

Controlling Pre-trained Language Models for Grade-Specific Text Simplification

RelVAE: Generative Pretraining for few-shot Visual Relationship Detection

KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model

What's left can't be right -- The remaining positional incompetence of contrastive vision-language models

Self-Supervised Pretraining for Heterogeneous Hypergraph Neural Networks

When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages

Do Language Models Learn Semantics of Code? A Case Study in Vulnerability Detection

Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training

PHD: Pixel-Based Language Modeling of Historical Documents

Self-Influence Guided Data Reweighting for Language Model Pre-training

Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew

Pushdown Layers: Encoding Recursive Structure in Transformer Language Models

ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing

Fragment-based Pretraining and Finetuning on Molecular Graphs

Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation

ViNT: A Foundation Model for Visual Navigation

Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution

RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling

Code-Switching with Word Senses for Pretraining in Neural Machine Translation

Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens

Unified Pretraining for Recommendation via Task Hypergraphs

Hierarchical Pretraining on Multimodal Electronic Health Records