语言大模型 / LLM - arXivDaily 专题

2408.03615 2024-10-22 cs.AI cs.CL 73%

Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks

Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, Liqiang Nie

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by NeurIPS 2024

2410.13212 2024-10-18 cs.LG cs.AI 73%

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations

Qian Tao, Wenyuan Yu, Jingren Zhou

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 12 pages, 4 figures

2410.11265 2024-10-16 cs.CL cs.AI 73%

In-Context Learning for Long-Context Sentiment Analysis on Infrastructure Project Opinions

Alireza Shamshiri, Kyeong Rok Ryu, June Young Park

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2410.01637 2024-10-04 cs.CL cs.LG 73%

On The Adaptation of Unlimiformer for Decoder-Only Transformers

Kian Ahrabian, Alon Benhaim, Barun Patra, Jay Pujara, Saksham Singhal, Xia Song

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments 8 pages, 6 figures

2410.01518 2024-10-04 cs.CL cs.LG 73%

InfiniPot: Infinite Context Processing on Memory-Constrained LLMs

Minsoo Kim, Kyuhong Shim, Jungwook Choi, Simyung Chang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2024 Main

2406.17419 2024-10-04 cs.CL cs.AI 73%

Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA

Minzheng Wang, Longze Chen, Cheng Fu, Shengyi Liao, Xinghua Zhang, Bingli Wu, Haiyang Yu, Nan Xu, Lei Zhang, Run Luo, Yunshui Li, Min Yang, Fei Huang, Yongbin Li

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2024 Main. We release our code and data publicly at https://github.com/MozerWang/Loong

2406.12335 2024-10-03 cs.CL cs.LG 73%

Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters

Zhiyu Guo, Hidetaka Kamigaito, Taro Watanabe

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Accepted at EMNLP 2024 (Main)

2409.12640 2024-09-23 cs.CL cs.LG 73%

Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries

Kiran Vodrahalli, Santiago Ontanon, Nilesh Tripuraneni, Kelvin Xu, Sanil Jain, Rakesh Shivanna, Jeffrey Hui, Nishanth Dikkala, Mehran Kazemi, Bahare Fatemi, Rohan Anil, Ethan Dyer, Siamak Shakeri, Roopali Vij, Harsh Mehta, Vinay Ramasesh, Quoc Le, Ed Chi, Yifeng Lu, Orhan Firat, Angeliki Lazaridou, Jean-Baptiste Lespiau, Nithya Attaluri, Kate Olszewska

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2309.00236 2024-09-19 cs.LG cs.CL cs.CR 73%

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

Luke Bailey, Euan Ong, Stuart Russell, Scott Emmons

专题命中长上下文与记忆：language model(abstract)；foundation model(abstract)；分类 cs.CL、cs.LG

Comments Project page at https://image-hijacks.github.io

2405.00790 2024-09-17 cs.AR cs.AI cs.DC cs.LG cs.PF 73%

SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators

Mohanad Odema, Luke Chen, Hyoukjun Kwon, Mohammad Abdullah Al Faruque

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments MICRO'24

2408.16967 2024-09-02 cs.CL cs.AI 73%

MemLong: Memory-Augmented Retrieval for Long Text Modeling

Weijie Liu, Zecheng Tang, Juntao Li, Kehai Chen, Min Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2408.12570 2024-08-23 cs.CL cs.LG 73%

Jamba-1.5: Hybrid Transformer-Mamba Models at Scale

Jamba Team, Barak Lenz, Alan Arazi, Amir Bergman, Avshalom Manevich, Barak Peleg, Ben Aviram, Chen Almagor, Clara Fridman, Dan Padnos, Daniel Gissin, Daniel Jannai, Dor Muhlgay, Dor Zimberg, Edden M Gerber, Elad Dolev, Eran Krakovsky, Erez Safahi, Erez Schwartz, Gal Cohen, Gal Shachaf, Haim Rozenblum, Hofit Bata, Ido Blass, Inbal Magar, Itay Dalmedigos, Jhonathan Osin, Julie Fadlon, Maria Rozman, Matan Danos, Michael Gokhman, Mor Zusman, Naama Gidron, Nir Ratner, Noam Gat, Noam Rozen, Oded Fried, Ohad Leshno, Omer Antverg, Omri Abend, Opher Lieber, Or Dagan, Orit Cohavi, Raz Alon, Ro'i Belson, Roi Cohen, Rom Gilad, Roman Glozman, Shahar Lev, Shaked Meirom, Tal Delbari, Tal Ness, Tomer Asida, Tom Ben Gal, Tom Braude, Uriya Pumerantz, Yehoshua Cohen, Yonatan Belinkov, Yuval Globerson, Yuval Peleg Levy, Yoav Shoham

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments Webpage: https://www.ai21.com/jamba

2311.09198 2024-08-15 cs.CL cs.AI 73%

Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training

Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, Yibo Liu, Qianguo Sun, Yuxin Liang, Hao Wang, Enming Zhang, Jiaxing Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024 main conference

2408.00274 2024-08-02 cs.CL cs.AI 73%

QUITO: Accelerating Long-Context Reasoning through Query-Guided Context Compression

Wenshan Wang, Yihang Wang, Yixing Fan, Huaming Liao, Jiafeng Guo

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2407.08608 2024-07-16 cs.LG cs.AI 73%

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, Tri Dao

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2404.02060 2024-06-13 cs.CL cs.AI 73%

Long-context LLMs Struggle with Long In-context Learning

Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, Wenhu Chen

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2406.06556 2024-06-12 cs.CL cs.AI 73%

Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach

Sambaran Bandyopadhyay, Himanshu Maheshwari, Anandhavelu Natarajan, Apoorv Saxena

专题命中长上下文与记忆：LLM(abstract)；prompting(abstract)；分类 cs.CL、cs.AI

2406.06124 2024-06-11 cs.CL cs.AI 73%

Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

Aadharsh Aadhithya A, Sachin Kumar S, Soman K. P

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 6 pages, 2 figures

2406.06025 2024-06-11 cs.SE cs.CL cs.LG 73%

RepoQA: Evaluating Long Context Code Understanding

Jiawei Liu, Jia Le Tian, Vijay Daita, Yuxiang Wei, Yifeng Ding, Yuhan Katherine Wang, Jun Yang, Lingming Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2310.10195 2024-06-07 cs.LG cs.CL 73%

AdaLomo: Low-memory Optimization with Adaptive Learning Rate

Kai Lv, Hang Yan, Qipeng Guo, Haijun Lv, Xipeng Qiu

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments ACL 2024 camera ready version

2406.00057 2024-06-06 cs.CL cs.LG 73%

Toward Conversational Agents with Context and Time Sensitive Long-term Memory

Nick Alonso, Tomás Figliolia, Anthony Ndirango, Beren Millidge

专题命中长上下文与记忆：language model(abstract)；prompting(abstract)；分类 cs.CL、cs.LG

2405.12981 2024-05-22 cs.LG cs.CL 73%

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

William Brandon, Mayank Mishra, Aniruddha Nrusimha, Rameswar Panda, Jonathan Ragan Kelly

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

2405.12035 2024-05-21 cs.AI cs.CL cs.IR 73%

KG-RAG: Bridging the Gap Between Knowledge and Creativity

Diego Sanmartin

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2404.10308 2024-04-17 cs.LG cs.AI 73%

Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs

Woomin Song, Seunghyuk Oh, Sangwoo Mo, Jaehyung Kim, Sukmin Yun, Jung-Woo Ha, Jinwoo Shin

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted to ICLR 2024. The first two authors contributed equally

2404.08674 2024-04-16 cs.CL cs.AI cs.HC 73%

Effects of Different Prompts on the Quality of GPT-4 Responses to Dementia Care Questions

Zhuochun Li, Bo Xie, Robin Hilsabeck, Alyssa Aguirre, Ning Zou, Zhimeng Luo, Daqing He

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2403.04787 2024-04-09 cs.CL cs.AI 73%

Ever-Evolving Memory by Blending and Refining the Past

Seo Hyun Kim, Keummin Ka, Yohan Jo, Seung-won Hwang, Dongha Lee, Jinyoung Yeo

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

Comments 17 pages, 4 figures, 7 tables

2310.03294 2024-04-02 cs.LG cs.AI cs.DC 73%

DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training

Dacheng Li, Rulin Shao, Anze Xie, Eric P. Xing, Xuezhe Ma, Ion Stoica, Joseph E. Gonzalez, Hao Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

2305.16344 2024-03-08 cs.CL cs.AI 73%

Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs

Chongjian Yue, Xinrun Xu, Xiaojun Ma, Lun Du, Hengyu Liu, Zhiming Ding, Yanbing Jiang, Shi Han, Dongmei Zhang

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI

2402.15923 2024-02-27 cs.LG cs.AI cs.MM 73%

Predicting Outcomes in Video Games with Long Short Term Memory Networks

Kittimate Chulajata, Sean Wu, Fabien Scalzo, Eun Sang Cha

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.AI、cs.LG

Comments 7 pages, 2 Figures, 2 Tables. Kittimate Chulajata and Sean Wu are considered co-first authors

2309.10400 2024-02-22 cs.CL cs.LG 73%

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training

Dawei Zhu, Nan Yang, Liang Wang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li

专题命中长上下文与记忆：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.LG

Comments ICLR 2024

1. 长上下文与记忆 4847 篇

Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations

In-Context Learning for Long-Context Sentiment Analysis on Infrastructure Project Opinions

On The Adaptation of Unlimiformer for Decoder-Only Transformers

InfiniPot: Infinite Context Processing on Memory-Constrained LLMs

Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA

Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters

Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators

MemLong: Memory-Augmented Retrieval for Long Text Modeling

Jamba-1.5: Hybrid Transformer-Mamba Models at Scale

Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training

QUITO: Accelerating Long-Context Reasoning through Query-Guided Context Compression

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

Long-context LLMs Struggle with Long In-context Learning

Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach

Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

RepoQA: Evaluating Long Context Code Understanding

AdaLomo: Low-memory Optimization with Adaptive Learning Rate

Toward Conversational Agents with Context and Time Sensitive Long-term Memory

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention

KG-RAG: Bridging the Gap Between Knowledge and Creativity

Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs

Effects of Different Prompts on the Quality of GPT-4 Responses to Dementia Care Questions

Ever-Evolving Memory by Blending and Refining the Past

DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training

Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs

Predicting Outcomes in Video Games with Long Short Term Memory Networks

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training