语言大模型 / LLM

Pruning Pre-trained Language Models Without Fine-Tuning

Ting Jiang, Deqing Wang, Fuzhen Zhuang, Ruobing Xie, Feng Xia

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to ACL 2023; Code and models are available at https://github.com/kongds/SMP

Language Model Analysis for Ontology Subsumption Inference

Yuan He, Jiaoyan Chen, Ernesto Jiménez-Ruiz, Hang Dong, Ian Horrocks

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at Findings of ACL 2023; OntoLAMA Datasets are available at: https://huggingface.co/datasets/krr-oxford/OntoLAMA (Huggingface) or https://doi.org/10.5281/zenodo.6480540 (Zenodo)

Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models

Joseph McDonald, Baolin Li, Nathan Frey, Devesh Tiwari, Vijay Gadepally, Siddharth Samsi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Journal ref Findings of the Association for Computational Linguistics: NAACL 2022

The MiniPile Challenge for Data-Efficient Language Models

Jean Kaddour

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Do we need Label Regularization to Fine-tune Pre-trained Language Models?

Ivan Kobyzev, Aref Jafari, Mehdi Rezagholizadeh, Tianda Li, Alan Do-Omri, Peng Lu, Pascal Poupart, Ali Ghodsi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Published at EACL 2023

A Cheaper and Better Diffusion Language Model with Soft-Masked Noise

Jiaao Chen, Aston Zhang, Mu Li, Alex Smola, Diyi Yang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Code is available at https://github.com/amazon-science/masked-diffusion-lm

Object-centric Inference for Language Conditioned Placement: A Foundation Model based Approach

Zhixuan Xu, Kechun Xu, Yue Wang, Rong Xiong

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments 6 pages, 6 figures

You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model

Shengkun Tang, Yaqing Wang, Zhenglun Kong, Tianchi Zhang, Yao Li, Caiwen Ding, Yanzhi Wang, Yi Liang, Dongkuan Xu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Dense Sparse Retrieval: Using Sparse Language Models for Inference Efficient Dense Retrieval

Daniel Campos, ChengXiang Zhai

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

AMOM: Adaptive Masking over Masking for Conditional Masked Language Model

Yisheng Xiao, Ruiyang Xu, Lijun Wu, Juntao Li, Tao Qin, Yan-Tie Liu, Min Zhang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by AAAI2023

Task-specific Compression for Multi-task Language Models using Attribution-based Pruning

Nakyeong Yang, Yunah Jang, Hwanhee Lee, Seohyeong Jung, Kyomin Jung

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 11 pages, 4 figures

Journal ref EACL 2023 Findings

PLOT: Prompt Learning with Optimal Transport for Vision-Language Models

Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, Kun Zhang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ICLR 2023, Spotlight

LaMPP: Language Models as Probabilistic Priors for Perception and Action

Belinda Z. Li, William Chen, Pratyusha Sharma, Jacob Andreas

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 12 pages, 4 tables, 4 figures

Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning

Malte Ostendorff, Georg Rehm

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling

Ahmed Elnaggar, Hazem Essam, Wafaa Salah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Charlotte Rochereau, Burkhard Rost

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 29 pages, 6 figures

In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models

Yukun Huang, Yanda Chen, Zhou Yu, Kathleen McKeown

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Building Metadata Inference Using a Transducer Based Language Model

David Waterworth, Subbu Sethuvenkatraman, Quan Z. Sheng

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Presented at First Australasia Symposium on Artificial Intelligence for the Environment (AI4Environment), 2022

sEHR-CE: Language modelling of structured EHR data for efficient and generalizable patient cohort expansion

Anna Munoz-Farre, Harry Rose, Sera Aylin Cakiroglu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Workshop on Learning from Time Series for Health, 36th Conference on Neural Information Processing Systems (NeurIPS 2022). arXiv admin note: text overlap with arXiv:2202.09318

Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference

Eric Mitchell, Joseph J. Noh, Siyan Li, William S. Armstrong, Ananth Agarwal, Patrick Liu, Chelsea Finn, Christopher D. Manning

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 16 pages. EMNLP 2022 Camera Ready. See https://ericmitchell.ai/emnlp-2022-concord/ for code and data

Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention

Zineng Tang, Jaemin Cho, Jie Lei, Mohit Bansal

专题命中 效率与部署 ：language model(title)；pretraining(abstract)；分类 cs.CL、cs.AI

Comments WACV 2023 (first two authors contributed equally)

Mask More and Mask Later: Efficient Pre-training of Masked Language Models by Disentangling the [MASK] Token

Baohao Liao, David Thulke, Sanjika Hewavitharana, Hermann Ney, Christof Monz

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Code available at: https://github.com/BaohaoLiao/3ml

Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks

Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, Reza Shokri

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.AI、cs.LG

Gradient Knowledge Distillation for Pre-trained Language Models

Lean Wang, Lei Li, Xu Sun

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted by NeurIPS ENLSP 2022 workshop(spotlight)

Recovering Private Text in Federated Learning of Language Models

Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, Danqi Chen

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2022. Code is publicly available at https://github.com/Princeton-SysML/FILM. v2 added discussion and evaluation of defenses

LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models

Mojan Javaheripi, Gustavo H. de Rosa, Subhabrata Mukherjee, Shital Shah, Tomasz L. Religa, Caio C. T. Mendes, Sebastien Bubeck, Farinaz Koushanfar, Debadeepta Dey

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

A Fast Post-Training Pruning Framework for Transformers

Woosuk Kwon, Sehoon Kim, Michael W. Mahoney, Joseph Hassoun, Kurt Keutzer, Amir Gholami

专题命中 效率与部署 ：post-training(title,abstract)；分类 cs.CL、cs.LG

Comments NeurIPS 2022

Post-Training Quantization for Energy Efficient Realization of Deep Neural Networks

Cecilia Latotzke, Batuhan Balim, Tobias Gemmeke

专题命中 效率与部署 ：post-training(title,abstract)；分类 cs.AI、cs.LG

Comments memory footprint, MSE, residuals, scale computation, channelwise, layerwise, word-length, bit-width

Spontaneous Emerging Preference in Two-tower Language Model

Zhengqi He, Taro Toyoizumi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Decoupled Context Processing for Context Augmented Language Modeling

Zonglin Li, Ruiqi Guo, Sanjiv Kumar

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Instance Regularization for Discriminative Language Model Pre-training

Zhuosheng Zhang, Hai Zhao, Ming Zhou

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2022