语言大模型 / LLM

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization

Aniruddha Nrusimha, Mayank Mishra, Naigang Wang, Dan Alistarh, Rameswar Panda, Yoon Kim

专题命中 效率与部署 ：language model(title,abstract)；post-training(abstract)；分类 cs.CL、cs.LG

SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLM

Quandong Wang, Yuxuan Yuan, Xiaoyu Yang, Ruike Zhang, Kang Zhao, Wei Liu, Jian Luan, Daniel Povey, Bin Wang

专题命中 效率与部署 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 5 figures, accepted by ECAI 2024

FASST: Fast LLM-based Simultaneous Speech Translation

Siqi Ouyang, Xi Xu, Chinmay Dandekar, Lei Li

专题命中 效率与部署 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models

Jan Hendrik Metzen, Piyapat Saranrittichai, Chaithanya Kumar Mummadi

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

Comments accepted at TMLR, Camera Ready Version

DESTEIN: Navigating Detoxification of Language Models via Universal Steering Pairs and Head-wise Activation Fusion

Yu Li, Han Jiang, Chuanyang Gong, Zhihua Wei

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Dissecting Language Models: Machine Unlearning via Selective Pruning

Nicholas Pochinkov, Nandi Schoots

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Cloud Atlas: Efficient Fault Localization for Cloud Systems using Language Models and Causal Insight

Zhiqiang Xie, Yujia Zheng, Lizi Ottens, Kun Zhang, Christos Kozyrakis, Jonathan Mace

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.AI、cs.LG

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks

Rui-Jie Zhu, Qihang Zhao, Guoqi Li, Jason K. Eshraghian

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted by TMLR

Promises, Outlooks and Challenges of Diffusion Language Modeling

Justin Deschenaux, Caglar Gulcehre

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training

Nan He, Weichen Xiong, Hanwen Liu, Yi Liao, Lei Ding, Kai Zhang, Guohua Tang, Xiao Han, Wei Yang

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 7 figures

Just read twice: closing the recall gap for recurrent language models

Simran Arora, Aman Timalsina, Aaryan Singhal, Benjamin Spector, Sabri Eyuboglu, Xinyi Zhao, Ashish Rao, Atri Rudra, Christopher Ré

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Efficient Training of Language Models with Compact and Consistent Next Token Distributions

Ashutosh Sathe, Sunita Sarawagi

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments ACL 2024

EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees

Yuhui Li, Fangyun Wei, Chao Zhang, Hongyang Zhang

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Integrating Pre-Trained Language Model with Physical Layer Communications

Ju-Hyung Lee, Dong-Ho Lee, Joohan Lee, Jay Pujara

专题命中 效率与部署 ：language model(title,abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, Yongqiang Ma

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 13 pages, accepted to ACL 2024 System Demonstration Track

Speech Prefix-Tuning with RNNT Loss for Improving LLM Predictions

Murali Karthick Baskar, Andrew Rosenberg, Bhuvana Ramabhadran, Neeraj Gaur, Zhong Meng

专题命中 效率与部署 ：LLM(title,abstract)；prompting(abstract)；分类 cs.CL、cs.AI

From Words to Worlds: Transforming One-line Prompt into Immersive Multi-modal Digital Stories with Communicative LLM Agent

Samuel S. Sohn, Danrui Li, Sen Zhang, Che-Jui Chang, Mubbasir Kapadia

专题命中 效率与部署 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 13 figures

SDQ: Sparse Decomposed Quantization for LLM Inference

Geonhwa Jeong, Po-An Tsai, Stephen W. Keckler, Tushar Krishna

专题命中 效率与部署 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Preprint

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jin Chen, Jingyang Yuan, Junjie Qiu, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruizhe Pan, Runxin Xu, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Size Zheng, T. Wang, Tian Pei, Tian Yuan, Tianyu Sun, W. L. Xiao, Wangding Zeng, Wei An, Wen Liu, Wenfeng Liang, Wenjun Gao, Wentao Zhang, X. Q. Li, Xiangyue Jin, Xianzu Wang, Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojin Shen, Xiaokang Chen, Xiaosha Chen, Xiaotao Nie, Xiaowen Sun, Xiaoxiang Wang, Xin Liu, Xin Xie, Xingkai Yu, Xinnan Song, Xinyi Zhou, Xinyu Yang, Xuan Lu, Xuecheng Su, Y. Wu, Y. K. Li, Y. X. Wei, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Li, Yaohui Wang, Yi Zheng, Yichao Zhang, Yiliang Xiong, Yilong Zhao, Ying He, Ying Tang, Yishi Piao, Yixin Dong, Yixuan Tan, Yiyuan Liu, Yongji Wang, Yongqiang Guo, Yuchen Zhu, Yuduan Wang, Yuheng Zou, Yukun Zha, Yunxian Ma, Yuting Yan, Yuxiang You, Yuxuan Liu, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhen Huang, Zhen Zhang, Zhenda Xie, Zhewen Hao, Zhihong Shao, Zhiniu Wen, Zhipeng Xu, Zhongyu Zhang, Zhuoshu Li, Zihan Wang, Zihui Gu, Zilin Li, Ziwei Xie

专题命中 效率与部署 ：language model(title,abstract)；SFT(abstract)；分类 cs.CL、cs.AI

Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference

Harry Dong, Xinyu Yang, Zhenyu Zhang, Zhangyang Wang, Yuejie Chi, Beidi Chen

专题命中 效率与部署 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Escalation Risks from Language Models in Military and Diplomatic Decision-Making

Juan-Pablo Rivera, Gabriel Mukobi, Anka Reuel, Max Lamparth, Chandler Smith, Jacquelyn Schneider

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 10 pages body, 57 pages appendix, 46 figures, 11 tables

Journal ref The 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT 24), June 3-6, 2024, Rio de Janeiro, Brazil

Flexible, Model-Agnostic Method for Materials Data Extraction from Text Using General Purpose Language Models

Maciej P. Polak, Shrey Modi, Anna Latosinska, Jinming Zhang, Ching-Wen Wang, Shaonan Wang, Ayan Deep Hazra, Dane Morgan

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 4 figures

Journal ref Digital Discovery, 2024, 3, 1221-1235

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference

Bowen Zhao, Hannaneh Hajishirzi, Qingqing Cao

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ICML 2024 Oral; code available at https://github.com/ROIM1998/APT

LLM and GNN are Complementary: Distilling LLM for Multimodal Graph Learning

Junjie Xu, Zongyu Wu, Minhua Lin, Xiang Zhang, Suhang Wang

专题命中 效率与部署 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Reinforcing Language Agents via Policy Optimization with Action Decomposition

Muning Wen, Ziyu Wan, Weinan Zhang, Jun Wang, Ying Wen

专题命中 效率与部署 ：language agent(title,abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 24 pages with 9 pages are main context

RecycleGPT: An Autoregressive Language Model with Recyclable Module

Yufan Jiang, Qiaozhi He, Xiaomin Zhuang, Zhihua Wu, Kunpeng Wang, Wenlai Zhao, Guangwen Yang

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments Technical Report

SUTRA: Scalable Multilingual Language Model Architecture

Abhijit Bendale, Michael Sapienza, Steven Ripplinger, Simon Gibbs, Jaewon Lee, Pranav Mistry

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuning

Shibo Jie, Yehui Tang, Ning Ding, Zhi-Hong Deng, Kai Han, Yunhe Wang

专题命中 效率与部署 ：prompting(title,abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ICML2024

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models

Qingqing Cao, Sewon Min, Yizhong Wang, Hannaneh Hajishirzi

专题命中 效率与部署 ：language model(title,abstract)；large language model(abstract)；分类 cs.CL、cs.AI

Comments ICLR 2024 camera-ready version

CHAI: Clustered Head Attention for Efficient LLM Inference

Saurabh Agarwal, Bilge Acun, Basil Hosmer, Mostafa Elhoushi, Yejin Lee, Shivaram Venkataraman, Dimitris Papailiopoulos, Carole-Jean Wu

专题命中 效率与部署 ：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG