语言大模型 / LLM - arXivDaily 专题

2305.14281 2023-10-20 cs.CL cs.CV 79%

Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining

Emanuele Bugliarello, Aida Nematzadeh, Lisa Anne Hendricks

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments EMNLP 2023

2305.14838 2023-10-17 cs.CL cs.SD eess.AS 79%

ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation

Chenyang Le, Yao Qian, Long Zhou, Shujie Liu, Yanmin Qian, Michael Zeng, Xuedong Huang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments NeurIPS 2023, Poster

2212.04755 2023-10-17 cs.CL 79%

From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader

Weiwen Xu, Xin Li, Wenxuan Zhang, Meng Zhou, Wai Lam, Luo Si, Lidong Bing

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to NeurIPS 2023

2308.12966 2023-10-16 cs.CV cs.CL 79%

Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, Jingren Zhou

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Code, demo and models are available at https://github.com/QwenLM/Qwen-VL

2307.15933 2023-10-16 cs.CL 79%

GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning

Soumyadeep Roy, Jonas Wallat, Sowmya S Sundaram, Wolfgang Nejdl, Niloy Ganguly

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 12 pages including appendix. Accepted for publication at 26th European Conference on Artificial Intelligence ECAI 2023

Journal ref Frontiers in Artificial Intelligence and Applications, Volume 372: ECAI 2023

2303.16887 2023-10-06 cs.CV cs.LG 79%

Towards Understanding the Effect of Pretraining Label Granularity

Guan Zhe Hong, Yin Cui, Ariel Fuxman, Stanley H. Chan, Enming Luo

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2309.17123 2023-10-02 cs.CV cs.LG 79%

Reconstruction of Patient-Specific Confounders in AI-based Radiologic Image Interpretation using Generative Pretraining

Tianyu Han, Laura Žigutytė, Luisa Huck, Marc Huppertz, Robert Siepmann, Yossi Gandelsman, Christian Blüthgen, Firas Khader, Christiane Kuhl, Sven Nebelung, Jakob Kather, Daniel Truhn

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2211.05750 2023-09-26 cs.CL 79%

Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control

Xiang Fan, Yiwei Lyu, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL Findings 2023

2309.08520 2023-09-18 cs.LG 79%

Scaling Laws for Sparsely-Connected Foundation Models

Elias Frantar, Carlos Riquelme, Neil Houlsby, Dan Alistarh, Utku Evci

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.LG

2308.09308 2023-09-18 cs.IR cs.CL 79%

Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification

Chenyu Zhao, Yunjiang Jiang, Yiming Qiu, Han Zhang, Wen-Yun Yang

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages, 2 figures; accepted by CIKM2023

2308.15670 2023-09-06 cs.CV cs.AI 79%

Multimodal Foundation Models For Echocardiogram Interpretation

Matthew Christensen, Milos Vukadinovic, Neal Yuan, David Ouyang

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2308.14050 2023-08-29 cs.CV cs.AI 79%

PECon: Contrastive Pretraining to Enhance Feature Alignment between CT and EHR Data for Improved Pulmonary Embolism Diagnosis

Santosh Sanjeev, Salwa K. Al Khatib, Mai A. Shaaban, Ibrahim Almakky, Vijay Ram Papineni, Mohammad Yaqub

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

2308.08129 2023-08-17 cs.LG q-bio.QM 79%

Is Self-Supervised Pretraining Good for Extrapolation in Molecular Property Prediction?

Shun Takashige, Masatoshi Hanai, Toyotaro Suzumura, Limin Wang, Kenjiro Taura

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2308.07146 2023-08-15 cs.CV cs.AI cs.MM 79%

CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation

Hongguang Zhu, Yunchao Wei, Xiaodan Liang, Chunjie Zhang, Yao Zhao

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted by ICCV 2023. Code: https://github.com/KevinLight831/CTP

2308.04551 2023-08-10 eess.IV cs.CV cs.LG 79%

Improving Medical Image Classification in Noisy Labels Using Only Self-supervised Pretraining

Bidur Khanal, Binod Bhattarai, Bishesh Khanal, Cristian A. Linte

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Accepted at MICCAI 2023 DEMI Workshop

2307.10457 2023-08-09 cs.CL 79%

Improving the Reusability of Pre-trained Language Models in Real-world Applications

Somayeh Ghanbarzadeh, Hamid Palangi, Yan Huang, Radames Cruz Moreno, Hamed Khanpour

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted as a long paper and awarded as the BEST Resaerch Paper in IEEE IRI'23 (IEEE 24th International conference on Information Reuse and Integrationfor Data Science)

2307.01759 2023-08-08 cs.CV cs.LG eess.IV 79%

Pretraining is All You Need: A Multi-Atlas Enhanced Transformer Framework for Autism Spectrum Disorder Classification

Lucas Mahler, Qi Wang, Julius Steiglechner, Florian Birk, Samuel Heczko, Klaus Scheffler, Gabriele Lohmann

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.00083 2023-08-02 cs.CL cs.IR 79%

In-Context Retrieval-Augmented Language Models

Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication in Transactions of the Association for Computational Linguistics (TACL). pre-MIT Press publication version

2307.13560 2023-08-01 cs.CL 79%

XDLM: Cross-lingual Diffusion Language Model for Machine Translation

Linyao Chen, Aosong Feng, Boming Yang, Zihui Li

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.CL

2106.11483 2023-07-27 cs.CL 79%

A Comprehensive Comparison of Pre-training Language Models

Tong Guo

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2209.10901 2023-07-20 cs.LG 79%

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning

Manuel Goulão, Arlindo L. Oliveira

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2307.07487 2023-07-17 cs.CV cs.LG 79%

DreamTeacher: Pretraining Image Backbones with Deep Generative Models

Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Seung Wook Kim, Karsten Kreis, Antonio Torralba, Sanja Fidler

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments Project page: https://research.nvidia.com/labs/toronto-ai/DreamTeacher/

2306.17649 2023-07-11 cs.CL 79%

Biomedical Language Models are Robust to Sub-optimal Tokenization

Bernal Jiménez Gutiérrez, Huan Sun, Yu Su

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments BioNLP @ ACL 2023

2307.03327 2023-07-10 cs.LG eess.SP 79%

Encoder-Decoder Networks for Self-Supervised Pretraining and Downstream Signal Bandwidth Regression on Digital Antenna Arrays

Rajib Bhattacharjea, Nathan West

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2307.01609 2023-07-06 cs.CL 79%

A Language Model for Grammatical Error Correction in L2 Russian

Nikita Remnev, Sergei Obiedkov, Ekaterina Rakhilina, Ivan Smirnov, Anastasia Vyrenkova

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2305.09790 2023-06-28 q-bio.BM cs.LG 79%

Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation

Cuong Q. Nguyen, Dante Pertusi, Kim M. Branson

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments ICML 2023 Workshop on Computational Biology

2302.05895 2023-06-27 cs.CL 79%

Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues

Chuyuan Li, Patrick Huber, Wen Xiao, Maxime Amblard, Chloé Braud, Giuseppe Carenini

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref Findings of the Association for Computational Linguistics: EACL 2023 (2023) 2562--2579

2303.15693 2023-06-14 cs.CV cs.LG eess.IV 79%

Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks

Masataka Kawai, Noriaki Ota, Shinsuke Yamaoka

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments 20 pages, 6 figures

2306.06892 2023-06-13 cs.CL 79%

On the N-gram Approximation of Pre-trained Language Models

Aravind Krishnan, Jesujoba Alabi, Dietrich Klakow

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at Interspeech 2023

2306.06598 2023-06-13 cs.CL 79%

RoBERTweet: A BERT Language Model for Romanian Tweets

Iulian-Marius Tăiatu, Andrei-Marius Avram, Dumitru-Clementin Cercel, Florin Pop

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted at NLDB2023

1. 预训练与数据 12429 篇

Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining

ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation

From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader

Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot Learning

Towards Understanding the Effect of Pretraining Label Granularity

Reconstruction of Patient-Specific Confounders in AI-based Radiologic Image Interpretation using Generative Pretraining

Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control

Scaling Laws for Sparsely-Connected Foundation Models

Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification

Multimodal Foundation Models For Echocardiogram Interpretation

PECon: Contrastive Pretraining to Enhance Feature Alignment between CT and EHR Data for Improved Pulmonary Embolism Diagnosis

Is Self-Supervised Pretraining Good for Extrapolation in Molecular Property Prediction?

CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation

Improving Medical Image Classification in Noisy Labels Using Only Self-supervised Pretraining

Improving the Reusability of Pre-trained Language Models in Real-world Applications

Pretraining is All You Need: A Multi-Atlas Enhanced Transformer Framework for Autism Spectrum Disorder Classification

In-Context Retrieval-Augmented Language Models

XDLM: Cross-lingual Diffusion Language Model for Machine Translation

A Comprehensive Comparison of Pre-training Language Models

Pretraining the Vision Transformer using self-supervised methods for vision based Deep Reinforcement Learning

DreamTeacher: Pretraining Image Backbones with Deep Generative Models

Biomedical Language Models are Robust to Sub-optimal Tokenization

Encoder-Decoder Networks for Self-Supervised Pretraining and Downstream Signal Bandwidth Regression on Digital Antenna Arrays

A Language Model for Grammatical Error Correction in L2 Russian

Molecule-Morphology Contrastive Pretraining for Transferable Molecular Representation

Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues

Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks

On the N-gram Approximation of Pre-trained Language Models

RoBERTweet: A BERT Language Model for Romanian Tweets