语言大模型 / LLM - arXivDaily 专题

2101.00434 2021-06-01 cs.CL 57%

Coreference Resolution without Span Representations

Yuval Kirstain, Ori Ram, Omer Levy

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments Accepted to ACL 2021

2105.14450 2021-06-01 cs.DC cs.LG cs.PF 57%

Maximizing Parallelism in Distributed Training for Huge Neural Networks

Zhengda Bian, Qifan Xu, Boxiang Wang, Yang You

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments Technical Report of NUS HPC-AI Lab (https://ai.comp.nus.edu.sg). The leading two authors have equal contributions

2105.05944 2021-05-14 cs.LG 57%

Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decay

Hsiang-Yun Sherry Chien, Javier S. Turek, Nicole Beckage, Vy A. Vo, Christopher J. Honey, Ted L. Willke

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments 16 pages, 10 figures

2104.14314 2021-04-30 cs.CL 57%

MOROCCO: Model Resource Comparison Framework

Valentin Malykh, Alexander Kukushkin, Ekaterina Artemova, Vladislav Mikhailov, Maria Tikhonova, Tatiana Shavrina

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

2002.09018 2021-03-08 cs.LG math.OC stat.ML 57%

Scalable Second Order Optimization for Deep Learning

Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, Yoram Singer

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments 24 pages, Code available here: https://bit.ly/3uXXtKy

2011.07831 2021-02-24 cs.LG cs.NE 57%

Learning Associative Inference Using Fast Weight Memory

Imanol Schlag, Tsendsuren Munkhdalai, Jürgen Schmidhuber

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

2008.02496 2021-02-03 cs.CL 57%

ConvBERT: Improving BERT with Span-based Dynamic Convolution

Zihang Jiang, Weihao Yu, Daquan Zhou, Yunpeng Chen, Jiashi Feng, Shuicheng Yan

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments 17 pages

2101.09788 2021-01-26 cs.CL 57%

Evaluating Models of Robust Word Recognition with Serial Reproduction

Stephan C. Meylan, Sathvik Nair, Thomas L. Griffiths

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Journal ref Cognition Volume 210, May 2021, 104553

2004.05150 2020-12-03 cs.CL 57%

Longformer: The Long-Document Transformer

Iz Beltagy, Matthew E. Peters, Arman Cohan

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments Version 2 introduces the Longformer-Encoder-Decoder (LED) model

2006.16981 2020-11-17 cs.LG cs.NE stat.ML 57%

Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules

Sarthak Mittal, Alex Lamb, Anirudh Goyal, Vikram Voleti, Murray Shanahan, Guillaume Lajoie, Michael Mozer, Yoshua Bengio

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments ICML 2020

2011.02886 2020-11-06 cs.LG 57%

Short-Term Memory Optimization in Recurrent Neural Networks by Autoencoder-based Initialization

Antonio Carta, Alessandro Sperduti, Davide Bacciu

专题命中长上下文与记忆：pretraining(abstract)；分类 cs.LG

Comments Accepted at NeurIPS 2020 workshop "Beyond Backpropagation: Novel Ideas for Training Neural Architectures"

2010.11351 2020-10-23 cs.CL 57%

Latte-Mix: Measuring Sentence Semantic Similarity with Latent Categorical Mixtures

M. Li, H. Bai, L. Tan, K. Xiong, M. Li, J. Lin

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

2010.10176 2020-10-21 cs.CL cs.IR 57%

Individual corpora predict fast memory retrieval during reading

Markus J. Hofmann, Lara Müller, Andre Rölke, Ralph Radach, Chris Biemann

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments Proceedings of the 6th workshop on Cognitive Aspects of the Lexicon (CogALex-VI), Barcelona, Spain, December 12, 2020; accepted manuscript; 11 pages, 2 figures, 4 Tables

2010.08892 2020-10-20 cs.CL 57%

Mixed-Lingual Pre-training for Cross-lingual Summarization

Ruochen Xu, Chenguang Zhu, Yu Shi, Michael Zeng, Xuedong Huang

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments Accepted at Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) 2020

2010.06447 2020-10-15 cs.CL 57%

Pagsusuri ng RNN-based Transfer Learning Technique sa Low-Resource Language

Dan John Velasco

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments 5 pages, 3 tables, 1 figure. in Filipino language; typos corrected, rephrased sentences, thoughts and results unchanged

2007.12723 2020-10-09 cs.LG cs.NE stat.ML 57%

Online Spatio-Temporal Learning in Deep Neural Networks

Thomas Bohnstingl, Stanisław Woźniak, Wolfgang Maass, Angeliki Pantazi, Evangelos Eleftheriou

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments Main manuscript: 9 pages, 3 figures, 1 table, Supplementary notes: 13 pages

2009.14783 2020-10-01 cs.DC cs.LG 57%

HetSeq: Distributed GPU Training on Heterogeneous Infrastructure

Yifan Ding, Nicholas Botzer, Tim Weninger

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments 7 pages, 3 tables, 2 figures

1811.04577 2020-08-25 cs.CL 57%

Forecasting People's Needs in Hurricane Events from Social Network

Long Nguyen, Zhou Yang, Jia Li, Guofeng Cao, Fang Jin

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

2006.10220 2020-06-23 cs.LG stat.ML 57%

I-BERT: Inductive Generalization of Transformer to Arbitrary Context Lengths

Hyoungwook Nam, Seung Byum Seo, Vikram Sharma Mailthody, Noor Michael, Lan Li

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments Submitted to NeurIPS2020

2002.11338 2020-05-27 cs.CV cs.LG cs.NE 57%

Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units

Zhanzhan Cheng, Yunlu Xu, Mingjian Cheng, Yu Qiao, Shiliang Pu, Yi Niu, Fei Wu

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

2005.08199 2020-05-26 cs.CL q-bio.NC 57%

How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?

Gantavya Bhatt, Hritik Bansal, Rishubh Singh, Sumeet Agarwal

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments 11 pages, 5 figures (including appendix); to appear at ACL SRW 2020

2005.10043 2020-05-21 cs.CL 57%

Leveraging Graph to Improve Abstractive Multi-Document Summarization

Wei Li, Xinyan Xiao, Jiachen Liu, Hua Wu, Haifeng Wang, Junping Du

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments Accepted by ACL2020

2005.06579 2020-05-15 cs.CL 57%

Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encoding

Xinya Du, Claire Cardie

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments Accepted to ACL 2020 (long papers), 12 pages

1910.02054 2020-05-14 cs.LG cs.DC stat.ML 57%

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

2005.00572 2020-05-05 cs.CL eess.AS 57%

Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition

Hu Hu, Rui Zhao, Jinyu Li, Liang Lu, Yifan Gong

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments Accepted by ICASSP 2020

2004.13336 2020-05-05 cs.DC cs.LG stat.ML 57%

Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training

Yuanzhong Xu, HyoukJoong Lee, Dehao Chen, Hongjun Choi, Blake Hechtman, Shibo Wang

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments 12 pages, 23 figures, 1 table

2004.14080 2020-04-30 cs.CL 57%

Modeling Long Context for Task-Oriented Dialogue State Generation

Jun Quan, Deyi Xiong

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

Comments ACL 2020

2004.05249 2020-04-14 cs.SE cs.LG cs.PL 57%

Sequence Model Design for Code Completion in the Modern IDE

Gareth Ari Aye, Gail E. Kaiser

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

1905.11286 2020-02-10 cs.LG stat.ML 57%

Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks

Boris Ginsburg, Patrice Castonguay, Oleksii Hrinchuk, Oleksii Kuchaiev, Vitaly Lavrukhin, Ryan Leary, Jason Li, Huyen Nguyen, Yang Zhang, Jonathan M. Cohen

专题命中长上下文与记忆：language model(abstract)；分类 cs.LG

Comments Preprint, under review

1909.01792 2020-01-30 cs.CL 57%

Mogrifier LSTM

Gábor Melis, Tomáš Kočiský, Phil Blunsom

专题命中长上下文与记忆：language model(abstract)；分类 cs.CL

1. 长上下文与记忆 4789 篇

Coreference Resolution without Span Representations

Maximizing Parallelism in Distributed Training for Huge Neural Networks

Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decay

MOROCCO: Model Resource Comparison Framework

Scalable Second Order Optimization for Deep Learning

Learning Associative Inference Using Fast Weight Memory

ConvBERT: Improving BERT with Span-based Dynamic Convolution

Evaluating Models of Robust Word Recognition with Serial Reproduction

Longformer: The Long-Document Transformer

Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules

Short-Term Memory Optimization in Recurrent Neural Networks by Autoencoder-based Initialization

Latte-Mix: Measuring Sentence Semantic Similarity with Latent Categorical Mixtures

Individual corpora predict fast memory retrieval during reading

Mixed-Lingual Pre-training for Cross-lingual Summarization

Pagsusuri ng RNN-based Transfer Learning Technique sa Low-Resource Language

Online Spatio-Temporal Learning in Deep Neural Networks

HetSeq: Distributed GPU Training on Heterogeneous Infrastructure

Forecasting People's Needs in Hurricane Events from Social Network

I-BERT: Inductive Generalization of Transformer to Arbitrary Context Lengths

Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units

How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?

Leveraging Graph to Improve Abstractive Multi-Document Summarization

Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encoding

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition

Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training

Modeling Long Context for Task-Oriented Dialogue State Generation

Sequence Model Design for Code Completion in the Modern IDE

Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks

Mogrifier LSTM