语言大模型 / LLM - arXivDaily 专题

2407.12391 2024-07-18 cs.DC cs.AI 85%

LLM Inference Serving: Survey of Recent Advances and Opportunities

Baolin Li, Yankai Jiang, Vijay Gadepally, Devesh Tiwari

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2407.09486 2024-07-16 cs.DC cs.AI 85%

ENOVA: Autoscaling towards Cost-effective and Stable Serverless LLM Serving

Tao Huang, Pengfei Chen, Kyoka Gong, Jocky Hawk, Zachary Bright, Wenxin Xie, Kecheng Huang, Zhi Ji

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2407.04051 2024-07-12 cs.SD cs.AI eess.AS 85%

FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs

Keyu An, Qian Chen, Chong Deng, Zhihao Du, Changfeng Gao, Zhifu Gao, Yue Gu, Ting He, Hangrui Hu, Kai Hu, Shengpeng Ji, Yabin Li, Zerui Li, Heng Lu, Haoneng Luo, Xiang Lv, Bin Ma, Ziyang Ma, Chongjia Ni, Changhe Song, Jiaqi Shi, Xian Shi, Hao Wang, Wen Wang, Yuxuan Wang, Zhangyu Xiao, Zhijie Yan, Yexin Yang, Bin Zhang, Qinglin Zhang, Shiliang Zhang, Nan Zhao, Siqi Zheng

专题命中效率与部署：foundation model(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Work in progress. Authors are listed in alphabetical order by family name

2407.00010 2024-07-02 cs.DC cs.AI 85%

Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads

Grant Wilkins, Srinivasan Keshav, Richard Mortier

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2401.05391 2024-06-25 cs.AR cs.AI 85%

Efficient LLM inference solution on Intel GPU

Hui Wu, Yi Gan, Feng Yuan, Jing Ma, Wei Zhu, Yutao Xu, Hong Zhu, Yuhua Zhu, Xiaoli Liu, Jinghui Gu, Peng Zhao

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

2406.13399 2024-06-21 cs.AI 85%

VELO: A Vector Database-Assisted Cloud-Edge Collaborative LLM QoS Optimization Framework

Zhi Yao, Zhiqing Tang, Jiong Lou, Ping Shen, Weijia Jia

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments to be published in IEEE ICWS 2024

2406.11674 2024-06-18 cs.CL 85%

Endor: Hardware-Friendly Sparse Format for Offloaded LLM Inference

Donghyeon Joo, Ramyad Hadidi, Soheil Feizi, Bahar Asgari

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 14 pages, 16 figures

2406.11257 2024-06-18 cs.LG 85%

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking

Wenshuo Li, Xinghao Chen, Han Shu, Yehui Tang, Yunhe Wang

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments ICML 2024 oral

2406.08765 2024-06-14 cs.LG 85%

LLM-based Knowledge Pruning for Time Series Data Analytics on Edge-computing Devices

Ruibing Jin, Qing Xu, Min Wu, Yuecong Xu, Dan Li, Xiaoli Li, Zhenghua Chen

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 12 pages, 5 figures

2406.07056 2024-06-12 cs.CL 85%

Effectively Compress KV Heads for LLM

Hao Yu, Zelan Yang, Shen Li, Yong Li, Jianxin Wu

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2406.07007 2024-06-12 cs.CL 85%

Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference

Jihwan Bang, Juntae Lee, Kyuhong Shim, Seunghan Yang, Simyung Chang

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments ACL 2024 Main

2305.12307 2024-06-12 cs.CL 85%

OntoType: Ontology-Guided and Pre-Trained Language Model Assisted Fine-Grained Entity Typing

Tanay Komarlu, Minhao Jiang, Xuan Wang, Jiawei Han

专题命中效率与部署：language model(title,abstract)；LLM(abstract)；prompting(abstract)；分类 cs.CL

2402.01620 2024-06-11 cs.CL 85%

MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models

Justin Chih-Yao Chen, Swarnadeep Saha, Elias Stengel-Eskin, Mohit Bansal

专题命中效率与部署：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.CL

Comments ICML 2024 (Camera-ready); First two authors contributed equally; GitHub: https://github.com/dinobby/MAGDi

2406.03853 2024-06-07 cs.CL 85%

Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism

Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ACL 2024 (Findings)

2312.07104 2024-06-07 cs.AI cs.PL 85%

SGLang: Efficient Execution of Structured Language Model Programs

Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, Ying Sheng

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；prompting(abstract)；分类 cs.AI

2405.17233 2024-06-04 cs.LG 85%

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs

Haoyu Wang, Bei Liu, Hang Shao, Bo Xiao, Ke Zeng, Guanglu Wan, Yanmin Qian

专题命中效率与部署：post-training(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

2405.19635 2024-05-31 cs.CL 85%

GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient Cloud-edge Collaboration LLM Deployment

Yao Yao, Zuchao Li, Hai Zhao

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

2312.06353 2024-05-28 cs.LG cs.DC 85%

Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes

Zhen Qin, Daoyuan Chen, Bingchen Qian, Bolin Ding, Yaliang Li, Shuiguang Deng

专题命中效率与部署：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.LG

Comments Accepted to ICML 2024. 25 pages, 14 figures, 7 tables. Codes are available at https://github.com/alibaba/FederatedScope/tree/FedKSeed

2402.05099 2024-05-14 cs.LG 85%

Hydragen: High-Throughput LLM Inference with Shared Prefixes

Jordan Juravsky, Bradley Brown, Ryan Ehrlich, Daniel Y. Fu, Christopher Ré, Azalia Mirhoseini

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2405.07212 2024-05-14 cs.NE cs.AI 85%

Enhancing Decision-Making in Optimization through LLM-Assisted Inference: A Neural Networks Perspective

Gaurav Singh, Kavitesh Kumar Bali

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted IJCNN

2405.06275 2024-05-13 cs.CL 85%

Pruning as a Domain-specific LLM Extractor

Nan Zhang, Yanchi Liu, Xujiang Zhao, Wei Cheng, Runxue Bao, Rui Zhang, Prasenjit Mitra, Haifeng Chen

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments NAACL 2024 Findings

2403.19838 2024-05-10 cs.CV cs.AI 85%

Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving

Akshay Gopalkrishnan, Ross Greer, Mohan Trivedi

专题命中效率与部署：language model(title,abstract)；LLM(abstract)；large language model(abstract)；分类 cs.AI

Comments 9 pages, 3 figures, Accepted at CVPR 2024 Vision and Language for Autonomous Driving and Robotics Workshop

2405.02749 2024-05-07 cs.LG 85%

Sub-goal Distillation: A Method to Improve Small Language Agents

Maryam Hashemzadeh, Elias Stengel-Eskin, Sarath Chandar, Marc-Alexandre Cote

专题命中效率与部署：language agent(title)；LLM(abstract)；large language model(abstract)；language model(abstract)

2310.19102 2024-04-17 cs.LG 85%

Atom: Low-bit Quantization for Efficient and Accurate LLM Serving

Yilong Zhao, Chien-Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, Baris Kasikci

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

2404.07470 2024-04-12 cs.CL 85%

Scalable Language Model with Generalized Continual Learning

Bohao Peng, Zhuotao Tian, Shu Liu, Mingchang Yang, Jiaya Jia

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；SLM(abstract)；分类 cs.CL

Comments The Twelfth International Conference on Learning Representations

2404.03977 2024-04-08 cs.CL 85%

SEME at SemEval-2024 Task 2: Comparing Masked and Generative Language Models on Natural Language Inference for Clinical Trials

Mathilde Aguiar, Pierre Zweigenbaum, Nona Naderi

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；prompting(abstract)；分类 cs.CL

2404.00640 2024-04-03 cs.SE cs.LG 85%

Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs

Shiwen Shan, Yintong Huo, Yuxin Su, Yichen Li, Dan Li, Zibin Zheng

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 13 pages, accepted by ISSTA 2024 (The 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis)

2403.18647 2024-04-02 cs.CL 85%

SDSAT: Accelerating LLM Inference through Speculative Decoding with Semantic Adaptive Tokens

Chengbo Liu, Yong Zhu

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 12 pages, 7 figures

2403.20306 2024-04-01 cs.AI cs.AR cs.DC 85%

Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference

Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Inigo Goiri, Josep Torrellas

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 6 pages, 15 figures

2402.04527 2024-03-20 cs.IR cs.AI 85%

RA-Rec: An Efficient ID Representation Alignment Framework for LLM-based Recommendation

Xiaohan Yu, Li Zhang, Xin Zhao, Yue Wang, Zhongrui Ma

专题命中效率与部署：LLM(title,abstract)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 10 pages

1. 效率与部署 22368 篇

LLM Inference Serving: Survey of Recent Advances and Opportunities

ENOVA: Autoscaling towards Cost-effective and Stable Serverless LLM Serving

FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs

Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads

Efficient LLM inference solution on Intel GPU

VELO: A Vector Database-Assisted Cloud-Edge Collaborative LLM QoS Optimization Framework

Endor: Hardware-Friendly Sparse Format for Offloaded LLM Inference

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking

LLM-based Knowledge Pruning for Time Series Data Analytics on Edge-computing Devices

Effectively Compress KV Heads for LLM

Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference

OntoType: Ontology-Guided and Pre-Trained Language Model Assisted Fine-Grained Entity Typing

MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models

Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism

SGLang: Efficient Execution of Structured Language Model Programs

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs

GKT: A Novel Guidance-Based Knowledge Transfer Framework For Efficient Cloud-edge Collaboration LLM Deployment

Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes

Hydragen: High-Throughput LLM Inference with Shared Prefixes

Enhancing Decision-Making in Optimization through LLM-Assisted Inference: A Neural Networks Perspective

Pruning as a Domain-specific LLM Extractor

Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving

Sub-goal Distillation: A Method to Improve Small Language Agents

Atom: Low-bit Quantization for Efficient and Accurate LLM Serving

Scalable Language Model with Generalized Continual Learning

SEME at SemEval-2024 Task 2: Comparing Masked and Generative Language Models on Natural Language Inference for Clinical Trials

Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs

SDSAT: Accelerating LLM Inference through Speculative Decoding with Semantic Adaptive Tokens

Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference

RA-Rec: An Efficient ID Representation Alignment Framework for LLM-based Recommendation