语言大模型 / LLM - arXivDaily 专题

2207.14251 2023-03-27 cs.CL 79%

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments We received a criticism regarding the validity of the causal formulation in this paper. We will address them in an upcoming version

2303.13076 2023-03-24 cs.CV cs.AI 79%

CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching

Xiaoshi Wu, Feng Zhu, Rui Zhao, Hongsheng Li

专题命中预训练与数据：prompting(title,abstract)；分类 cs.AI

Comments 11 pages, 4 figures. Accepted by CVPR 2023

2211.03053 2023-03-15 cs.CL 79%

Suffix Retrieval-Augmented Language Modeling

Zecheng Wang, Yik-Cheung Tam

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 5 pages, 1 figure. Submitted to ICASSP 2023

2303.03457 2023-03-08 cs.CL 79%

Spelling convention sensitivity in neural language models

Elizabeth Nielsen, Christo Kirov, Brian Roark

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref EACL Findings 2023

2303.03144 2023-03-07 cs.MM cs.CL cs.SD eess.AS 79%

IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining

Chihaya Matsuhira, Marc A. Kastner, Takahiro Komamizu, Takatsugu Hirayama, Keisuke Doman, Yasutomo Kawanishi, Ichiro Ide

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages, 8 figures, 5 Tables

2210.07468 2023-03-07 cs.CL 79%

Transparency Helps Reveal When Language Models Learn Meaning

Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Accepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2023. Author's final version (pre-MIT Press publication)

2303.01566 2023-03-06 stat.ML cs.LG math.ST stat.TH 79%

On the Provable Advantage of Unsupervised Pretraining

Jiawei Ge, Shange Tang, Jianqing Fan, Chi Jin

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2206.13602 2023-03-02 cs.LG q-bio.QM 79%

Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching

Shengchao Liu, Hongyu Guo, Jian Tang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.14389 2023-03-01 cs.CL 79%

Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context

Alexandre Pasquiou, Yair Lakretz, Bertrand Thirion, Christophe Pallier

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments 19 pages, 8 figures, 10 pages of Appendix, 5 appendix figures

2302.14141 2023-03-01 cs.LG 79%

Linear pretraining in recurrent mixture density networks

Hubert Normandin-Taillon, Frédéric Godin, Chun Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.09483 2023-02-21 cs.LG 79%

Why Is Public Pretraining Necessary for Private Model Training?

Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Thakurta, Lun Wang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2302.08427 2023-02-17 cs.CV cs.AI 79%

Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies

Emma Sarfati, Alexandre Bone, Marc-Michel Rohe, Pietro Gori, Isabelle Bloch

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.AI

Comments Accepted at IEEE ISBI 2023

2210.07587 2023-02-14 cs.CL 79%

ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining

Ranran Haoran Zhang, Aysa Xuemo Fan, Rui Zhang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

Comments Accepted by EACL 2023

2206.03826 2023-02-14 cs.LG cs.CV cs.NE stat.ML 79%

Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Tasks

Jiachun Pan, Pan Zhou, Shuicheng Yan

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2208.03219 2023-02-07 cs.CL 79%

Construction of English Resume Corpus and Test with Pre-trained Language Models

Chengguang Gan, Tatsunori Mori

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments The Association for Natural Language Processing in Japan. Submit in NLP2023

2302.01483 2023-02-06 cs.LG cs.SD eess.AS 79%

SPADE: Self-supervised Pretraining for Acoustic DisEntanglement

John Harvill, Jarred Barber, Arun Nair, Ramin Pishehvar

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2207.04993 2023-02-01 cs.CL 79%

Embedding Recycling for Language Models

Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug Downey

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments EACL Findings 2023

2301.02111 2023-01-06 cs.CL cs.SD eess.AS 79%

Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers

Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, Furu Wei

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Comments Working in progress

2301.00447 2023-01-03 cs.CV cs.LG 79%

Image To Tree with Recursive Prompting

James Batten, Matthew Sinclair, Ben Glocker, Michiel Schaap

专题命中预训练与数据：prompting(title,abstract)；分类 cs.LG

Comments 12 pages, 5 figures

2207.13988 2023-01-03 cs.CL 79%

Sequence to sequence pretraining for a less-resourced Slovenian language

Matej Ulčar, Marko Robnik-Šikonja

专题命中预训练与数据：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 19 pages

2212.08967 2022-12-20 cs.AI 79%

Foundation models in brief: A historical, socio-technical focus

Johannes Schneider

专题命中预训练与数据：foundation model(title,abstract)；分类 cs.AI

2212.08619 2022-12-19 cs.CL cs.CR 79%

Planting and Mitigating Memorized Content in Predictive-Text Language Models

C. M. Downey, Wei Dai, Huseyin A. Inan, Kim Laine, Saurabh Naik, Tomasz Religa

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2212.08321 2022-12-19 eess.AS cs.CL 79%

Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language

Yusuke Yasuda, Tomoki Toda

专题命中预训练与数据：language model(title)；pretraining(abstract)；分类 cs.CL

Journal ref IEEE Journal of Selected Topics in Signal Processing (Volume: 16, Issue: 6, October 2022)

2203.10929 2022-12-08 cs.CL 79%

General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining

Qi Lv, Ziqiang Cao, Lei Geng, Chunhui Ai, Xu Yan, Guohong Fu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2211.13337 2022-12-07 cs.LG 79%

Multi-Environment Pretraining Enables Transfer to Action Limited Datasets

David Venuto, Sherry Yang, Pieter Abbeel, Doina Precup, Igor Mordatch, Ofir Nachum

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

2211.03545 2022-12-06 eess.AS cs.CL cs.SD 79%

ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech

Xiaoran Fan, Chao Pang, Tian Yuan, He Bai, Renjie Zheng, Pengfei Zhu, Shuohuan Wang, Junkun Chen, Zeyu Chen, Liang Huang, Yu Sun, Hua Wu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.CL

2212.01681 2022-12-06 cs.CL cs.MA 79%

Language Models as Agent Models

Jacob Andreas

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

2207.08824 2022-11-30 q-bio.QM cs.LG 79%

Energy-Motivated Equivariant Pretraining for 3D Molecular Graphs

Rui Jiao, Jiaqi Han, Wenbing Huang, Yu Rong, Yang Liu

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments AAAI 2023

2211.14429 2022-11-29 physics.chem-ph cs.LG q-bio.BM 79%

Supervised Pretraining for Molecular Force Fields and Properties Prediction

Xiang Gao, Weihao Gao, Wenzhi Xiao, Zhirui Wang, Chong Wang, Liang Xiang

专题命中预训练与数据：pretraining(title,abstract)；分类 cs.LG

Comments AI4Science Workshop at NeurIPS 2022

2112.08462 2022-11-28 cs.CL 79%

Applying SoftTriple Loss for Supervised Language Model Fine Tuning

Witold Sosnowski, Anna Wroblewska, Piotr Gawrysiak

专题命中预训练与数据：language model(title,abstract)；分类 cs.CL

Journal ref 17th Conference on Computer Science and Intelligence Systems 2022. Series: ACSIS Annals of Computer Science and Information Systems

1. 预训练与数据 12429 篇

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching

Suffix Retrieval-Augmented Language Modeling

Spelling convention sensitivity in neural language models

IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretraining

Transparency Helps Reveal When Language Models Learn Meaning

On the Provable Advantage of Unsupervised Pretraining

Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching

Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context

Linear pretraining in recurrent mixture density networks

Why Is Public Pretraining Necessary for Private Model Training?

Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies

ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining

Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Tasks

Construction of English Resume Corpus and Test with Pre-trained Language Models

SPADE: Self-supervised Pretraining for Acoustic DisEntanglement

Embedding Recycling for Language Models

Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers

Image To Tree with Recursive Prompting

Sequence to sequence pretraining for a less-resourced Slovenian language

Foundation models in brief: A historical, socio-technical focus

Planting and Mitigating Memorized Content in Predictive-Text Language Models

Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language

General and Domain Adaptive Chinese Spelling Check with Error Consistent Pretraining

Multi-Environment Pretraining Enables Transfer to Action Limited Datasets

ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech

Language Models as Agent Models

Energy-Motivated Equivariant Pretraining for 3D Molecular Graphs

Supervised Pretraining for Molecular Force Fields and Properties Prediction

Applying SoftTriple Loss for Supervised Language Model Fine Tuning