语言大模型 / LLM

Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrieval

Libo Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments The experimental process and code have been uploaded to the Github repository, the link is: https://github.com/brucewang123456789/GeniusTrail/tree/main/Wormhole%20Memory%20Module

The Method for Storing Patterns in Neural Networks-Memorization and Recall of QR code Patterns-

Hiroshi Inazawa

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 14 pages, 6 figures

Training-Free Exponential Context Extension via Cascading KV Cache

Jeffrey Willette, Heejun Lee, Youngwan Lee, Myeongjae Jeon, Sung Ju Hwang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Robot Character Generation and Adaptive Human-Robot Interaction with Personality Shaping

Cheng Tang, Chao Tang, Steven Gong, Thomas M. Kwok, Yue Hu

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs

Alireza Rezazadeh, Zichao Li, Wei Wei, Yujia Bao

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

COLA: A Scalable Multi-Agent Framework For Windows UI Task Automation

Di Zhao, Longhui Ma, Siwei Wang, Miao Wang, Zhao Lv

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference

Guangtao Wang, Shubhangi Upasani, Chen Wu, Darshan Gandhi, Jonathan Li, Changran Hu, Bo Li, Urmish Thakker

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Towards Reliable Vector Database Management Systems: A Software Testing Roadmap for 2030

Shenao Wang, Yanjie Zhao, Yinglin Xie, Zhao Liu, Xinyi Hou, Quanchen Zou, Haoyu Wang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

LongSafety: Enhance Safety for Long-Context LLMs

Mianqiu Huang, Xiaoran Liu, Shaojun Zhou, Mozhi Zhang, Qipeng Guo, Linyang Li, Chenkun Tan, Yang Gao, Pengyu Wang, Linlin Li, Qun Liu, Yaqian Zhou, Xipeng Qiu, Xuanjing Huang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Implicit In-context Learning

Zhuowei Li, Zihao Xu, Ligong Han, Yunhe Gao, Song Wen, Di Liu, Hao Wang, Dimitris N. Metaxas

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

The Role of Sparsity for Length Generalization in Transformers

Noah Golowich, Samy Jelassi, David Brandfonbrener, Sham M. Kakade, Eran Malach

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

On the Robustness of Transformers against Context Hijacking for Linear Classification

Tianle Li, Chenyang Zhang, Xingwu Chen, Yuan Cao, Difan Zou

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

MoBA: Mixture of Block Attention for Long-Context LLMs

Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong, Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, Zhiqi Huang, Huan Yuan, Suting Xu, Xinran Xu, Guokun Lai, Yanru Chen, Huabin Zheng, Junjie Yan, Jianlin Su, Yuxin Wu, Neo Y. Zhang, Zhilin Yang, Xinyu Zhou, Mingxing Zhang, Jiezhong Qiu

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 15 pages

RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations

Zunhai Su, Zhe Chen, Wang Shen, Hanyu Wei, Linge Li, Huangqi Yu, Kehong Yuan

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Profiling Apple Silicon Performance for ML Training

Dahua Feng, Zhiming Xu, Rongxiang Wang, Felix Xiaozhu Lin

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Adjoint sharding for very long context training of state space models

Xingzi Xu, Amir Tavanaei, Kavosh Asadi, Karim Bouyarmane

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Casevo: A Cognitive Agents and Social Evolution Simulator

Zexun Jiang, Yafang Shi, Maoxu Li, Hongjiang Xiao, Yunxiao Qin, Qinglan Wei, Ye Wang, Yuan Zhang

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Ultra-High-Definition Dynamic Multi-Exposure Image Fusion via Infinite Pixel Learning

Xingchi Chen, Zhuoran Zheng, Xuerui Li, Yuying Chen, Shu Wang, Wenqi Ren

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Interactive AI with Retrieval-Augmented Generation for Next Generation Networking

Ruichen Zhang, Hongyang Du, Yinqiu Liu, Dusit Niyato, Jiawen Kang, Sumei Sun, Xuemin Shen, H. Vincent Poor

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 10 pages, 4 figures

Eigen Attention: Attention in Low-Rank Space for KV Cache Compression

Utkarsh Saxena, Gobinda Saha, Sakshi Choudhary, Kaushik Roy

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 12 page, 6 figures, 6 tables

Positive Experience Reflection for Agents in Interactive Text Environments

Philip Lippmann, Matthijs T. J. Spaan, Jie Yang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To appear at NeurIPS 2024 Language Gamification workshop

IANUS: Integrated Accelerator based on NPU-PIM Unified Memory System

Minseok Seo, Xuan Truong Nguyen, Seok Joong Hwang, Yongkee Kwon, Guhyun Kim, Chanwook Park, Ilkon Kim, Jaehan Park, Jeongbin Kim, Woojae Shin, Jongsoon Won, Haerang Choi, Kyuyoung Kim, Daehan Kwon, Chunseok Jeong, Sangheon Lee, Yongseok Choi, Wooseok Byun, Seungcheol Baek, Hyuk-Jae Lee, John Kim

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments Updated version of the paper accepted to ASPLOS 2024

Journal ref ASPLOS 2024

Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and Transferability

Tsz Ting Chung, Leyang Cui, Lemao Liu, Xinting Huang, Shuming Shi, Dit-Yan Yeung

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 14 pages, 5 figures, 10 tables, EMNLP 2024 Findings

CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarios

Luning Wang, Shiyao Li, Xuefei Ning, Zhihang Yuan, Shengen Yan, Guohao Dai, Yu Wang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 4th NeurIPS Efficient Natural Language and Speech Processing Workshop (ENLSP-IV 2024)

Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach

Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMNLP 2024 industry track

Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning

Chengsong Huang, Langlin Huang, Jiaxin Huang

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Composite Learning Units: Generalized Learning Beyond Parameter Updates to Transform LLMs into Adaptive Reasoners

Santosh Kumar Radha, Oktay Goktas

专题命中 长上下文与记忆 ：large language model(abstract)；language model(abstract)；分类 cs.CL、cs.AI、cs.LG

Optimal Workload Placement on Multi-Instance GPUs

Bekir Turkkan, Pavankumar Murali, Pavithra Harsha, Rohan Arora, Gerard Vanloo, Chandra Narayanaswami

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 14 pages

Language to Map: Topological map generation from natural language path instructions

Hideki Deguchi, Kazuki Shibata, Shun Taguchi

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments 7 pages, 7 figures. Accepted to IEEE International Conference on Robotics and Automation (ICRA) 2024

BMX: Entropy-weighted Similarity and Semantic-enhanced Lexical Search

Xianming Li, Julius Lipp, Aamir Shakir, Rui Huang, Jing Li

专题命中 长上下文与记忆 ：LLM(abstract)；large language model(abstract)；language model(abstract)

Comments correct the affiliation order