语言大模型 / LLM - arXivDaily 专题

2409.00088 2024-09-17 cs.CL 83%

On-Device Language Models: A Comprehensive Review

Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang, Xin Gao, Qi Cai, Ziyuan Ling

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 38 pages, 6 figures

2402.17988 2024-09-06 cs.PL cs.LG cs.SE 83%

Constrained Decoding for Fill-in-the-Middle Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars

Daniel Melcer, Nathan Fulton, Sanjay Krishna Gouda, Haifeng Qian

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.LG

Comments 10 pages, Code available at https://github.com/amazon-science/incremental-parsing

2408.13010 2024-08-26 cs.LG stat.AP 83%

A Web-Based Solution for Federated Learning with LLM-Based Automation

Chamith Mawela, Chaouki Ben Issaid, Mehdi Bennis

专题命中效率与部署：LLM(title,abstract)；language model(abstract)；分类 cs.LG

2408.08146 2024-08-16 cs.CL 83%

KOALA: Enhancing Speculative Decoding for LLM via Multi-Layer Draft Heads with Adversarial Learning

Kaiqi Zhang, Jing Zhao, Rui Chen

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2407.19610 2024-07-30 cs.AI 83%

Mixture of Modular Experts: Distilling Knowledge from a Multilingual Teacher into Specialized Modular Language Models

Mohammed Al-Maamari, Mehdi Ben Amor, Michael Granitzer

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.AI

Comments Preprint

2407.19185 2024-07-30 cs.CV cs.AI 83%

LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models

Ruiyi Zhang, Yufan Zhou, Jian Chen, Jiuxiang Gu, Changyou Chen, Tong Sun

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.AI

Comments NeurIPS 2024 Under Review

2407.17353 2024-07-25 cs.LG 83%

Scalify: scale propagation for efficient low-precision LLM training

Paul Balança, Sam Hosegood, Carlo Luschi, Andrew Fitzgibbon

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments 11 pages, 5 figures, ICML 2024 WANT workshop

2405.15341 2024-07-23 cs.AI cs.CV 83%

V-Zen: Efficient GUI Understanding and Precise Grounding With A Novel Multimodal LLM

Abdur Rahman, Rajat Chawla, Muskaan Kumar, Arkajit Datta, Adarsh Jha, Mukunda NS, Ishaan Bhola

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 12 pages, 5 figures, 3 tables

2403.07678 2024-07-22 cs.CL cs.CY 83%

MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions

Vjosa Preniqi, Iacopo Ghinassi, Julia Ive, Charalampos Saitis, Kyriaki Kalimeri

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Journal ref ACM 4th International Conference on Information Technology for Social Good (GoodIT 2024)

2407.09241 2024-07-15 cs.CL 83%

The Sociolinguistic Foundations of Language Modeling

Jack Grieve, Sara Bartl, Matteo Fuoli, Jason Grafmiller, Weihang Huang, Alejandro Jawerbaum, Akira Murakami, Marcus Perlman, Dana Roemling, Bodo Winter

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2407.05040 2024-07-09 cs.SE cs.LG 83%

Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruning

Yun-Da Tsai, Mingjie Liu, Haoxing Ren

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.LG

2311.18237 2024-07-03 cs.CV cs.LG 83%

Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models

Raviteja Vemulapalli, Hadi Pouransari, Fartash Faghri, Sachin Mehta, Mehrdad Farajtabar, Mohammad Rastegari, Oncel Tuzel

专题命中效率与部署：foundation model(title,abstract)；pretraining(abstract)；分类 cs.LG

Comments International Conference on Machine Learning, 2024

2407.00945 2024-07-02 cs.LG 83%

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs

Enshu Liu, Junyi Zhu, Zinan Lin, Xuefei Ning, Matthew B. Blaschko, Shengen Yan, Guohao Dai, Huazhong Yang, Yu Wang

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.LG

2406.19954 2024-07-01 cs.CL cs.HC cs.SD eess.AS 83%

BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5

Zhehuai Chen, He Huang, Oleksii Hrinchuk, Krishna C. Puvvada, Nithin Rao Koluguri, Piotr Żelasko, Jagadeesh Balam, Boris Ginsburg

专题命中效率与部署：language model(title,abstract)；LLM(abstract)；分类 cs.CL

2310.09234 2024-06-27 cs.IR cs.AI 83%

ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction

Jianghao Lin, Bo Chen, Hangyu Wang, Yunjia Xi, Yanru Qu, Xinyi Dai, Kangning Zhang, Ruiming Tang, Yong Yu, Weinan Zhang

专题命中效率与部署：language model(title,abstract)；pretraining(abstract)；分类 cs.AI

Comments Accepted by WWW 2024

2401.14555 2024-06-26 cs.CV cs.LG 83%

Revisiting Active Learning in the Era of Vision Foundation Models

Sanket Rajan Gupte, Josiah Aklilu, Jeffrey J. Nirschl, Serena Yeung-Levy

专题命中效率与部署：foundation model(title,abstract)；language model(abstract)；分类 cs.LG

Comments Accepted to TMLR

2309.00789 2024-06-26 cs.CL 83%

LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models

Abhishek Arora, Melissa Dell

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2406.14150 2024-06-21 cs.LG 83%

Multi-modal Transfer Learning between Biological Foundation Models

Juan Jose Garau-Luis, Patrick Bordes, Liam Gonzalez, Masa Roller, Bernardo P. de Almeida, Lorenz Hexemer, Christopher Blum, Stefan Laurent, Jan Grzegorzewski, Maren Lang, Thomas Pierrot, Guillaume Richard

专题命中效率与部署：foundation model(title)；large language model(abstract)；language model(abstract)；分类 cs.LG

2405.12532 2024-06-06 cs.CL 83%

PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference

Dongjie Yang, XiaoDong Han, Yan Gao, Yao Hu, Shilin Zhang, Hai Zhao

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Accepted by ACL 2024

2402.00856 2024-06-06 cs.CL 83%

Towards Efficient Exact Optimization of Language Model Alignment

Haozhe Ji, Cheng Lu, Yilin Niu, Pei Ke, Hongning Wang, Jun Zhu, Jie Tang, Minlie Huang

专题命中效率与部署：language model(title,abstract)；preference optimization(abstract)；分类 cs.CL

Comments 24 pages, 9 figures

Journal ref Forty-first International Conference on Machine Learning (ICML 2024)

2403.03507 2024-06-04 cs.LG 83%

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, Yuandong Tian

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.LG

Comments ICML 2024 (Oral)

2405.17741 2024-05-29 cs.AI 83%

LoRA-Switch: Boosting the Efficiency of Dynamic LLM Adapters via System-Algorithm Co-design

Rui Kong, Qiyang Li, Xinyu Fang, Qingtian Feng, Qingfeng He, Yazhu Dong, Weijun Wang, Yuanchun Li, Linghe Kong, Yunxin Liu

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

2405.16241 2024-05-28 cs.CR cs.AI 83%

FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference

Chenqi Lin, Tianshi Xu, Zebin Yang, Runsheng Wang, Ru Huang, Meng Li

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments 6 pages, DAC2024

2405.14023 2024-05-24 cs.LG 83%

WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response

Tianrong Zhang, Bochuan Cao, Yuanpu Cao, Lu Lin, Prasenjit Mitra, Jinghui Chen

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.LG

2405.13155 2024-05-24 cs.LG 83%

ReALLM: A general framework for LLM compression and fine-tuning

Louis Leconte, Lisa Bedin, Van Minh Nguyen, Eric Moulines

专题命中效率与部署：LLM(title)；language model(abstract)；post-training(abstract)；分类 cs.LG

2404.03921 2024-05-16 cs.CL 83%

Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models

Bowen Zhang, Kehua Chang, Chunping Li

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted by ICIC 2024 (Oral)

2405.05254 2024-05-10 cs.CL 83%

You Only Cache Once: Decoder-Decoder Architectures for Language Models

Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, Furu Wei

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2401.01970 2024-05-07 cs.CV cs.AI 83%

FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding

Xingxing Zuo, Pouya Samangouei, Yunwen Zhou, Yan Di, Mingyang Li

专题命中效率与部署：foundation model(title,abstract)；language model(abstract)；分类 cs.AI

Comments Project page: https://xingxingzuo.github.io/fmgs

2405.02246 2024-05-06 cs.CV cs.AI 83%

What matters when building vision-language models?

Hugo Laurençon, Léo Tronchon, Matthieu Cord, Victor Sanh

专题命中效率与部署：language model(title,abstract)；large language model(abstract)；分类 cs.AI

2404.18057 2024-04-30 cs.CL 83%

Efficient LLM Inference with Kcache

Qiaozhi He, Zhihua Wu

专题命中效率与部署：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments Technical Report, 8 pages

1. 效率与部署 22456 篇

On-Device Language Models: A Comprehensive Review

Constrained Decoding for Fill-in-the-Middle Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars

A Web-Based Solution for Federated Learning with LLM-Based Automation

KOALA: Enhancing Speculative Decoding for LLM via Multi-Layer Draft Heads with Adversarial Learning

Mixture of Modular Experts: Distilling Knowledge from a Multilingual Teacher into Specialized Modular Language Models

LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models

Scalify: scale propagation for efficient low-precision LLM training

V-Zen: Efficient GUI Understanding and Precise Grounding With A Novel Multimodal LLM

MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions

The Sociolinguistic Foundations of Language Modeling

Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruning

Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models

Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs

BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5

ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction

Revisiting Active Learning in the Era of Vision Foundation Models

LinkTransformer: A Unified Package for Record Linkage with Transformer Language Models

Multi-modal Transfer Learning between Biological Foundation Models

PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference

Towards Efficient Exact Optimization of Language Model Alignment

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

LoRA-Switch: Boosting the Efficiency of Dynamic LLM Adapters via System-Algorithm Co-design

FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference

WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response

ReALLM: A general framework for LLM compression and fine-tuning

Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models

You Only Cache Once: Decoder-Decoder Architectures for Language Models

FMGS: Foundation Model Embedded 3D Gaussian Splatting for Holistic 3D Scene Understanding

What matters when building vision-language models?

Efficient LLM Inference with Kcache