语言大模型 / LLM

Module-wise Adaptive Distillation for Multimodality Foundation Models

Chen Liang, Jiahui Yu, Ming-Hsuan Yang, Matthew Brown, Yin Cui, Tuo Zhao, Boqing Gong, Tianyi Zhou

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.CL、cs.LG

Neural Language Model Pruning for Automatic Speech Recognition

Leonardo Emili, Thiago Fraga-Silva, Ernest Pusateri, Markus Nußbaum-Thom, Youssef Oualil

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 3 figures

RT-LM: Uncertainty-Aware Resource Management for Real-Time Inference of Language Models

Yufei Li, Zexin Li, Wei Yang, Cong Liu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted by RTSS 2023

3D Denoisers are Good 2D Teachers: Molecular Pretraining via Denoising and Cross-Modal Distillation

Sungjun Cho, Dae-Woong Jeong, Sung Moon Ko, Jinwoo Kim, Sehui Han, Seunghoon Hong, Honglak Lee, Moontae Lee

专题命中 效率与部署 ：pretraining(title,abstract)；分类 cs.AI、cs.LG

Comments 16 pages, 5 figures

EntropyRank: Unsupervised Keyphrase Extraction via Side-Information Optimization for Language Model-based Text Compression

Alexander Tsvetkov, Alon Kipnis

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Journal ref ICML 2023 Workshop Neural Compression: From Information Theory to Applications

BERT4CTR: An Efficient Framework to Combine Pre-trained Language Model with Non-textual Features for CTR Prediction

Dong Wang, Kavé Salamatian, Yunqing Xia, Weiwei Deng, Qi Zhiang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Bayesian Prompt Learning for Image-Language Model Generalization

Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat, Victor Guilherme Turrisi da Costa, Cees G. M. Snoek, Georgios Tzimiropoulos, Brais Martinez

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICCV 2023

Foundation Model is Efficient Multimodal Multitask Model Selector

Fanqing Meng, Wenqi Shao, Zhanglin Peng, Chonghe Jiang, Kaipeng Zhang, Yu Qiao, Ping Luo

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Membership Inference Attacks against Language Models via Neighbourhood Comparison

Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model

Yukyung Lee, Jina Kim, Pilsung Kang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Population Expansion for Training Language Models with Private Federated Learning

Tatsuki Koga, Congzheng Song, Martin Pelikan, Mona Chitnis

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference

Junhao Zheng, Qianli Ma, Shengjie Qiu, Yue Wu, Peitian Ma, Junlong Liu, Huawen Feng, Xichen Shang, Haibin Chen

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2023 (oral paper)

Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness

Eric Zelikman, Qian Huang, Percy Liang, Nick Haber, Noah D. Goodman

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model

Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Yang Yang, Hongyin Tang, Keqing He, Jiahao Liu, Jingang Wang, Shu Zhao, Peng Zhang, Jie Tang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments accepted for ACL 2023 industry track

Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method

Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Shu Zhao, Peng Zhang, Jie Tang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to Findings of ACL2023

Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization

Pengcheng He, Baolin Peng, Liyang Lu, Song Wang, Jie Mei, Yang Liu, Ruochen Xu, Hany Hassan Awadalla, Yu Shi, Chenguang Zhu, Wayne Xiong, Michael Zeng, Jianfeng Gao, Xuedong Huang

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 3 figures. Accepted as long paper in main conference of ACL 2023

A Scalable and Adaptive System to Infer the Industry Sectors of Companies: Prompt + Model Tuning of Generative Language Models

Lele Cao, Vilhelm von Ehrenheim, Astrid Berghult, Cecilia Henje, Richard Anselmo Stahl, Joar Wandborg, Sebastian Stan, Armin Catovic, Erik Ferm, Hannes Ingelhag

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by FinNLP (Financial Technology and Natural Language Processing) @ IJCAI2023 as long paper (8 pages and 8 figures)

A Quantitative Review on Language Model Efficiency Research

Meng Jiang, Hy Dang, Lingbo Tong

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 29 pages, 24 tables

Continual Task Allocation in Meta-Policy Network via Sparse Prompting

Yijun Yang, Tianyi Zhou, Jing Jiang, Guodong Long, Yuhui Shi

专题命中 效率与部署 ：prompting(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by ICML 2023

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods

Zhuo Zhang, Yuanhang Yang, Yong Dai, Lizhen Qu, Zenglin Xu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR

W. Ronny Huang, Hao Zhang, Shankar Kumar, Shuo-yiin Chang, Tara N. Sainath

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Interspeech 2023. First 3 authors contributed equally

Multi-armed bandits for resource efficient, online optimization of language model pre-training: the use case of dynamic masking

Iñigo Urteaga, Moulay-Zaïdane Draïdia, Tomer Lancewicki, Shahram Khadivi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Work accepted for publication at ACL Findings 2023. The code used for this study is publicly available at https://github.com/iurteaga/gp_ts_nlp

Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation

Minglun Han, Feilong Chen, Jing Shi, Shuang Xu, Bo Xu

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by INTERSPEECH 2023

PuMer: Pruning and Merging Tokens for Efficient Vision Language Models

Qingqing Cao, Bhargavi Paranjape, Hannaneh Hajishirzi

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2023 Main Conference

Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models

Bum Chul Kwon, Nandana Mihindukulasooriya

专题命中 效率与部署 ：foundation model(title)；language model(abstract)；分类 cs.CL、cs.AI

Comments ACL 2023 System Demonstrations, 9 pages, 3 figures

Building Transportation Foundation Model via Generative Graph Transformer

Xuhong Wang, Ding Wang, Liang Chen, Yilun Lin

专题命中 效率与部署 ：foundation model(title,abstract)；分类 cs.AI、cs.LG

Enhancing Cross-lingual Prompting with Dual Prompt Augmentation

Meng Zhou, Xin Li, Yue Jiang, Lidong Bing

专题命中 效率与部署 ：prompting(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2023 Findings

DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models

Xuxi Chen, Tianlong Chen, Weizhu Chen, Ahmed Hassan Awadallah, Zhangyang Wang, Yu Cheng

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted by ACL 2023

Enhancing Cross-lingual Natural Language Inference by Soft Prompting with Multilingual Verbalizer

Shuang Li, Xuming Hu, Aiwei Liu, Yawen Yang, Fukun Ma, Philip S. Yu, Lijie Wen

专题命中 效率与部署 ：prompting(title,abstract)；分类 cs.CL、cs.AI

Comments Accept at ACL2023

Lifting the Curse of Capacity Gap in Distilling Language Models

Chen Zhang, Yang Yang, Jiahao Liu, Jingang Wang, Yunsen Xian, Benyou Wang, Dawei Song

专题命中 效率与部署 ：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 17 pages, 6 figures, 13 tables, accepted to ACL 2023. Code is available at https://github.com/GeneZC/MiniMoE