语言大模型 / LLM

Zero-Shot Video Question Answering via Frozen Bidirectional Language Models

Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, Cordelia Schmid

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2022 Camera-Ready; Project Webpage: https://antoyang.github.io/frozenbilm.html; 25 pages; 5 figures

FairDistillation: Mitigating Stereotyping in Language Models

Pieter Delobelle, Bettina Berendt

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at ECML-PKDD 2022

Bayesian Neural Network Language Modeling for Speech Recognition

Boyang Xue, Shoukang Hu, Junhao Xu, Mengzhe Geng, Xunying Liu, Helen Meng

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

LM-CORE: Language Models with Contextually Relevant External Knowledge

Jivat Neet Kaur, Sumit Bhatia, Milan Aggarwal, Rachit Bansal, Balaji Krishnamurthy

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Published at Findings of NAACL, 2022

Efficient Fine-Tuning of Compressed Language Models with Learners

Danilo Vucetic, Mohammadreza Tayaranian, Maryam Ziaeefard, James J. Clark, Brett H. Meyer, Warren J. Gross

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 9 figures, 2 tables, presented at ICML 2022 workshop on Hardware-Aware Efficient Training (HAET 2022)

Language Model Cascades

David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-dickstein, Kevin Murphy, Charles Sutton

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Presented as spotlight at the Beyond Bases workshop at ICML 2022 (https://beyond-bayes.github.io)

Analyzing Bagging Methods for Language Models

Pranab Islam, Shaan Khosla, Arthur Lok, Mudit Saxena

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Knowledge Distillation of Transformer-based Language Models Revisited

Chengqiang Lu, Jianwei Zhang, Yunfei Chu, Zhengyu Chen, Jingren Zhou, Fei Wu, Haiqing Chen, Hongxia Yang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Continual Learning with Foundation Models: An Empirical Study of Latent Replay

Oleksiy Ostapenko, Timothee Lesort, Pau Rodríguez, Md Rifat Arefin, Arthur Douillard, Irina Rish, Laurent Charlin

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Space-Efficient Representation of Entity-centric Query Language Models

Christophe Van Gysel, Mirko Hannemann, Ernest Pusateri, Youssef Oualil, Ilya Oparin

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Interspeech '22

Scaling Language Model Size in Cross-Device Federated Learning

Jae Hun Ro, Theresa Breiner, Lara McConnaughey, Mingqing Chen, Ananda Theertha Suresh, Shankar Kumar, Rajiv Mathews

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Accelerating Inference and Language Model Fusion of Recurrent Neural Network Transducers via End-to-End 4-bit Quantization

Andrea Fasoli, Chia-Yu Chen, Mauricio Serrano, Swagath Venkataramani, George Saon, Xiaodong Cui, Brian Kingsbury, Kailash Gopalakrishnan

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 5 pages, 2 figures, 1 table. Paper accepted to Interspeech 2022

Bridging the Gap Between Training and Inference of Bayesian Controllable Language Models

Han Liu, Bingning Wang, Ting Yao, Haijin Liang, Jianjin Xu, Xiaolin Hu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Submitted to Neurips 2022

Causal Distillation for Language Models

Zhengxuan Wu, Atticus Geiger, Josh Rozner, Elisa Kreiss, Hanson Lu, Thomas Icard, Christopher Potts, Noah D. Goodman

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 7 pages, 2 figures

Journal ref NAACL 2022

Efficient Language Modeling with Sparse all-MLP

Ping Yu, Mikel Artetxe, Myle Ott, Sam Shleifer, Hongyu Gong, Ves Stoyanov, Xian Li

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

BabyBear: Cheap inference triage for expensive language models

Leila Khalili, Yao You, John Bohannon

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 7 pages, 6 figures

Discourse-Aware Soft Prompting for Text Generation

Marjan Ghazvininejad, Vladimir Karpukhin, Vera Gor, Asli Celikyilmaz

专题命中 效率与部署 ：prompting(title)；language model(abstract)；分类 cs.CL、cs.LG

Exploring Extreme Parameter Compression for Pre-trained Language Models

Yuxin Ren, Benyou Wang, Lifeng Shang, Xin Jiang, Qun Liu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted at ICLR2022. Code available at https://github.com/twinkle0331/Xcompression

Knowledge Distillation of Russian Language Models with Reduction of Vocabulary

Alina Kolesnikova, Yuri Kuratov, Vasily Konovalov, Mikhail Burtsev

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction

Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, Jing Shao

专题命中 效率与部署 ：prompting(title)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2022 main conference

Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal

Umang Gupta, Jwala Dhamala, Varun Kumar, Apurv Verma, Yada Pruksachatkun, Satyapriya Krishna, Rahul Gupta, Kai-Wei Chang, Greg Ver Steeg, Aram Galstyan

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments To appear in the Findings of ACL 2022

R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling

Xiang Hu, Haitao Mi, Zujie Wen, Yafang Wang, Yi Su, Jing Zheng, Gerard de Melo

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ACL-IJCNLP 2021

E-LANG: Energy-Based Joint Inferencing of Super and Swift Language Models

Mohammad Akbari, Amin Banitalebi-Dehkordi, Yong Zhang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ACL 2022

Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model

Hao Zhang, You-Chi Cheng, Shankar Kumar, W. Ronny Huang, Mingqing Chen, Rajiv Mathews

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2108.11943

Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

Yu Meng, Yunyi Zhang, Jiaxin Huang, Yu Zhang, Jiawei Han

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments WWW 2022. (Code: https://github.com/yumeng5/TopClus)

ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise

Minjia Zhang, Niranjan Uma Naresh, Yuxiong He

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Uncovering More Shallow Heuristics: Probing the Natural Language Inference Capacities of Transformer-Based Pre-Trained Language Models Using Syllogistic Patterns

Reto Gubelmann, Siegfried Handschuh

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

On the Compression of Natural Language Models

Saeed Damadi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression

Runxin Xu, Fuli Luo, Chengyu Wang, Baobao Chang, Jun Huang, Songfang Huang, Fei Huang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to AAAI 2022

Language Modelling via Learning to Rank

Arvid Frydenlund, Gagandeep Singh, Frank Rudzicz

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to AAAI22. Minor writing fixes