Online Scheduling for LLM Inference with KV Cache Constraints
在线调度用于具有KV缓存约束的LLM推理
Patrick Jaillet, Jiashuo Jiang, Konstantina Mellou, Marco Molinaro, Chara Podimata, Zijie Zhou
机构
*
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology(麻省理工学院电子工程与计算机科学系)
;
HKUST(香港科技大学)
;
Microsoft Research(微软研究院)
;
Sloan School of Management, Massachusetts Institute of Technology(斯隆管理学院,麻省理工学院)
;
Operations Research Center, Massachusetts Institute of Technology(运营研究中心,麻省理工学院)
专题命中
效率与部署
:LLM(title,abstract);large language model(abstract);language model(abstract);分类 cs.AI、cs.LG
MoST: Mixing Speech and Text with Modality-Aware Mixture of Experts
MoST:通过模态感知混合专家混合语音和文本
Yuxuan Lou, Kai Yang, Yang You
机构
*
School of Computer Science, National University of Singapore(新加坡国立大学计算机科学学院)
;
School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院)
专题命中
效率与部署
:LLM(abstract);large language model(abstract);language model(abstract);post-training(abstract)
Generative AI collective behavior needs an interactionist paradigm
生成式AI的集体行为需要一种互动主义范式
Laura Ferrarotti, Gian Maria Campedelli, Roberto Dessì, Andrea Baronchelli, Giovanni Iacca, Kathleen M. Carley, Alex Pentland, Joel Z. Leibo, James Evans, Bruno Lepri
机构
*
Fondazione Bruno Kessler(布雷诺·克塞尔基金会)
;
University of Trento(特伦托大学)
;
Not Diamond
;
City St. George’s University of London(伦敦圣乔治大学)
;
Carnegie Mellon University(卡内基梅隆大学)
;
Massachusetts Institute of Technology(麻省理工学院)
;
Stanford University(斯坦福大学)
;
Google DeepMind(谷歌DeepMind)
;
University of Chicago(芝加哥大学)
专题命中
效率与部署
:LLM(abstract);large language model(abstract);language model(abstract);分类 cs.AI、cs.LG
The Straight and Narrow: Do LLMs Possess an Internal Moral Path?
直行与狭窄:大型语言模型是否具有内在的道德路径?
Luoming Hu, Jingjie Zeng, Liang Yang, Hongfei Lin
机构
*
School of Future Technology, Dalian University of Technology, China(大连理工大学未来技术学院)
;
School of Computer Science and Technology, Dalian University of Technology, China(大连理工大学计算机科学与技术学院)
;
Key Laboratory of Social Computing and Cognitive Intelligence, Ministry of Education, China(教育部社会计算与认知智能重点实验室)
专题命中
效率与部署
:large language model(abstract);language model(abstract);分类 cs.CL
The State-of-the-Art in Lifelog Retrieval: A Review of Progress at the ACM Lifelog Search Challenge Workshop 2022-24
生命周期检索的最新进展:ACM生命周期检索挑战工作坊2022-24年进展综述
Allie Tran, Werner Bailer, Duc-Tien Dang-Nguyen, Graham Healy, Steve Hodges, Björn Þór Jónsson, Luca Rossetto, Klaus Schoeffmann, Minh-Triet Tran, Lucia Vadicamo, Cathal Gurrin
专题命中
效率与部署
:large language model(abstract);language model(abstract)