语言大模型 / LLM - arXivDaily 专题

2411.01142 2024-11-05 cs.DC cs.AI cs.LG 81%

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference

Xuanlin Jiang, Yang Zhou, Shiyi Cao, Ion Stoica, Minlan Yu

专题命中效率与部署：LLM(title,abstract)；分类 cs.AI、cs.LG

2401.01099 2024-10-28 eess.AS cs.AI cs.LG 81%

Efficient Parallel Audio Generation using Group Masked Language Modeling

Myeonghun Jeong, Minchan Kim, Joun Yeop Lee, Nam Soo Kim

专题命中效率与部署：language model(title,abstract)；分类 cs.AI、cs.LG

Comments This work has been submitted to the IEEE for possible publication

2405.15319 2024-10-23 cs.CL cs.AI 81%

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Wenyu Du, Tongxu Luo, Zihan Qiu, Zeyu Huang, Yikang Shen, Reynold Cheng, Yike Guo, Jie Fu

专题命中效率与部署：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments NeurIPS 2024 Spotlight

2410.14911 2024-10-22 cs.CV cs.AI cs.CL 81%

A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models

Yuhan Liang, Yijun Li, Yumeng Niu, Qianhe Shen, Hangyu Liu

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

2406.14683 2024-10-22 cs.LG cs.CL 81%

TAGLAS: An atlas of text-attributed graph datasets in the era of large graph and language models

Jiarui Feng, Hao Liu, Lecheng Kong, Mingfang Zhu, Yixin Chen, Muhan Zhang

专题命中效率与部署：language model(title)；foundation model(abstract)；分类 cs.CL、cs.LG

Comments Preprint

2410.14072 2024-10-21 cs.CV cs.AI cs.CL 81%

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers

Yuxin Wen, Qingqing Cao, Qichen Fu, Sachin Mehta, Mahyar Najibi

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

2406.00800 2024-10-18 cs.LG cs.AI 81%

MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization

Aozhong Zhang, Naigang Wang, Yanxia Deng, Xin Li, Zi Yang, Penghang Yin

专题命中效率与部署：post-training(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted by NeurIPS 2024

2401.06640 2024-10-18 cs.CL cs.AI 81%

Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently

Kanishka Misra, Allyson Ettinger, Kyle Mahowald

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 (main) camera-ready

2410.02952 2024-10-11 cs.CL cs.AI 81%

Visual Editing with LLM-based Tool Chaining: An Efficient Distillation Approach for Real-Time Applications

Oren Sultan, Alex Khasin, Guy Shiran, Asnat Greenstein-Messica, Dafna Shahaf

专题命中效率与部署：LLM(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024

2402.15020 2024-10-11 cs.LG cs.CL 81%

Towards Probabilistically-Sound Beam Search with Masked Language Models

Creston Brooks, Robert Calef, Charlie Cowen-Breen, Anna Sappington

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

2410.06441 2024-10-10 cs.LG cs.CL 81%

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models

Zeman Li, Xinwei Zhang, Peilin Zhong, Yuan Deng, Meisam Razaviyayn, Vahab Mirrokni

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

2408.16737 2024-10-10 cs.CL cs.AI 81%

Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling

Hritik Bansal, Arian Hosseini, Rishabh Agarwal, Vinh Q. Tran, Mehran Kazemi

专题命中效率与部署：LLM(title)；language model(abstract)；分类 cs.CL、cs.AI

2410.04962 2024-10-08 cs.CL cs.AI 81%

Activation Scaling for Steering and Interpreting Language Models

Niklas Stoehr, Kevin Du, Vésteinn Snæbjarnarson, Robert West, Ryan Cotterell, Aaron Schein

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Findings of the Association for Computational Linguistics: EMNLP 2024

2410.03040 2024-10-07 cs.CL cs.LG cs.NA math.NA 81%

Geometry is All You Need: A Unified Taxonomy of Matrix and Tensor Factorization for Compression of Generative Language Models

Mingxue Xu, Sadia Sharmin, Danilo P. Mandic

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

2410.03007 2024-10-07 eess.AS cs.AI cs.CL 81%

FastAdaSP: Multitask-Adapted Efficient Inference for Large Speech Language Model

Yichen Lu, Jiaqi Song, Chao-Han Huck Yang, Shinji Watanabe

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 Industry Track

2402.12821 2024-10-07 cs.CL cs.LG 81%

Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomy

Liyan Xu, Zhenlin Su, Mo Yu, Jin Xu, Jinho D. Choi, Jie Zhou, Fei Liu

专题命中效率与部署：LLM(title,abstract)；分类 cs.CL、cs.LG

Comments Accepted to EMNLP 2024 Findings

2410.01019 2024-10-03 cs.CL cs.LG 81%

Investigating the Synergistic Effects of Dropout and Residual Connections on Language Model Training

Qingyang Li, Weimao Ke

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

Comments 5 pages, 4 figures

2410.00519 2024-10-02 cs.CL cs.AI 81%

Exploring the Learning Capabilities of Language Models using LEVERWORLDS

Eitan Wagner, Amir Feder, Omri Abend

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

2409.10064 2024-09-17 cs.CL cs.AI cs.HC 81%

MindGuard: Towards Accessible and Sitgma-free Mental Health First Aid via Edge LLM

Sijie Ji, Xinzhe Zheng, Jiawei Sun, Renqi Chen, Wei Gao, Mani Srivastava

专题命中效率与部署：LLM(title,abstract)；分类 cs.CL、cs.AI

2401.04472 2024-09-09 cs.LG cs.AI cs.DC 81%

A Survey on Efficient Federated Learning Methods for Foundation Model Training

Herbert Woisetschläger, Alexander Isenko, Shiqiang Wang, Ruben Mayer, Hans-Arno Jacobsen

专题命中效率与部署：foundation model(title,abstract)；分类 cs.AI、cs.LG

Comments Accepted for publication at IJCAI 2024. Please cite the published paper via https://doi.org/10.24963/ijcai.2024/919

2403.06764 2024-09-04 cs.CV cs.AI cs.CL 81%

An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models

Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, Baobao Chang

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ECCV 2024 (Oral), code is released at https://github.com/pkunlp-icler/FastV,

2311.12023 2024-08-28 cs.CL cs.LG 81%

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning

Han Guo, Philip Greengard, Eric P. Xing, Yoon Kim

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

2405.02267 2024-08-27 cs.LG cs.CL 81%

Structural Pruning of Pre-trained Language Models via Neural Architecture Search

Aaron Klein, Jacek Golebiowski, Xingchen Ma, Valerio Perrone, Cedric Archambeau

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

2408.10473 2024-08-21 cs.CL cs.LG 81%

Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism

Guanchen Li, Xiandong Zhao, Lian Liu, Zeping Li, Dong Li, Lu Tian, Jie He, Ashish Sirasao, Emad Barsoum

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

2405.03133 2024-08-20 cs.CL cs.LG 81%

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training

Zexuan Zhong, Mengzhou Xia, Danqi Chen, Mike Lewis

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

Comments COLM 2024

2305.17100 2024-08-13 cs.CL cs.AI 81%

BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks

Kai Zhang, Rong Zhou, Eashan Adhikarla, Zhiling Yan, Yixin Liu, Jun Yu, Zhengliang Liu, Xun Chen, Brian D. Davison, Hui Ren, Jing Huang, Chen Chen, Yuyin Zhou, Sunyang Fu, Wei Liu, Tianming Liu, Xiang Li, Yong Chen, Lifang He, James Zou, Quanzheng Li, Hongfang Liu, Lichao Sun

专题命中效率与部署：foundation model(title,abstract)；分类 cs.CL、cs.AI

Comments Fix incorrect citations and add journal reference for the published version. Nat Med (2024)

2408.03964 2024-08-09 cs.NI cs.AI cs.LG 81%

Telecom Foundation Models: Applications, Challenges, and Future Trends

Tahar Zanouda, Meysam Masoudi, Fitsum Gaim Gebre, Mischa Dohler

专题命中效率与部署：foundation model(title,abstract)；分类 cs.AI、cs.LG

2407.21046 2024-08-01 cs.CL cs.LG 81%

Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines

Yuchen Li, Alexandre Kirchmeyer, Aashay Mehta, Yilong Qin, Boris Dadachev, Kishore Papineni, Sanjiv Kumar, Andrej Risteski

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.LG

Comments ICML 2024

2407.17060 2024-07-25 cs.CV cs.AI cs.CL eess.IV 81%

High Efficiency Image Compression for Large Visual-Language Models

Binzhe Li, Shurun Wang, Shiqi Wang, Yan Ye

专题命中效率与部署：language model(title,abstract)；分类 cs.CL、cs.AI

2402.16268 2024-07-19 cs.LG cs.AI cs.CY 81%

Foundation Model Transparency Reports

Rishi Bommasani, Kevin Klyman, Shayne Longpre, Betty Xiong, Sayash Kapoor, Nestor Maslej, Arvind Narayanan, Percy Liang

专题命中效率与部署：foundation model(title,abstract)；分类 cs.AI、cs.LG

Journal ref Published in AIES 2024

1. 效率与部署 22497 篇

NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference

Efficient Parallel Audio Generation using Group Masked Language Modeling

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models

TAGLAS: An atlas of text-attributed graph datasets in the era of large graph and language models

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers

MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization

Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently

Visual Editing with LLM-based Tool Chaining: An Efficient Distillation Approach for Real-Time Applications

Towards Probabilistically-Sound Beam Search with Masked Language Models

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models

Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling

Activation Scaling for Steering and Interpreting Language Models

Geometry is All You Need: A Unified Taxonomy of Matrix and Tensor Factorization for Compression of Generative Language Models

FastAdaSP: Multitask-Adapted Efficient Inference for Large Speech Language Model

Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomy

Investigating the Synergistic Effects of Dropout and Residual Connections on Language Model Training

Exploring the Learning Capabilities of Language Models using LEVERWORLDS

MindGuard: Towards Accessible and Sitgma-free Mental Health First Aid via Edge LLM

A Survey on Efficient Federated Learning Methods for Foundation Model Training

An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning

Structural Pruning of Pre-trained Language Models via Neural Architecture Search

Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training

BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks

Telecom Foundation Models: Applications, Challenges, and Future Trends

Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines

High Efficiency Image Compression for Large Visual-Language Models

Foundation Model Transparency Reports