语言大模型 / LLM - arXivDaily 专题

2505.20444 2025-10-09 cs.LG cs.CV 83%

HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models

Haoran Li, Yingjie Qin, Baoyuan Ou, Lai Xu, Ruiwen Xu

机构 * Carnegie Mellon University（卡内基梅隆大学）； Xiaohongshu Inc.（小红书公司）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.LG

Comments NeurIPS 2025

2510.03978 2025-10-07 cs.CV cs.CL 83%

No Tokens Wasted: Leveraging Long Context in Biomedical Vision-Language Models

Min Woo Sun, Alejandro Lozano, Javier Gamazo Tejero, Vishwesh Nath, Xiao Xiao Sun, James Burgess, Yuhui Zhang, Kun Yuan, Robert Tibshirani, Sean Huver, Serena Yeung-Levy

机构 * Stanford University（斯坦福大学）； NVIDIA USA（NVIDIA公司）

专题命中长上下文与记忆：language model(title,abstract)；pretraining(abstract)；分类 cs.CL

2502.01630 2025-09-26 cs.AI 83%

TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues

Yubin Ge, Salvatore Romeo, Jason Cai, Raphael Shu, Monica Sunkara, Yassine Benajiba, Yi Zhang

机构 * University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）； Amazon Web Services（亚马逊网络服务）

专题命中长上下文与记忆：LLM(title,abstract)；prompting(abstract)；分类 cs.AI

Comments Accepted at ACL 2025 Findings

2504.03770 2025-07-18 cs.CR cs.AI 83%

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

Yi Nian, Shenzhe Zhu, Yuehan Qin, Li Li, Ziyi Wang, Chaowei Xiao, Yue Zhao

机构 * University of Southern California（南加州大学）； University of Toronto（多伦多大学）； University of Maryland（马里兰大学）； University of Wisconsin-Madison（威斯康星大学麦迪逊分校）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.AI

2412.16545 2025-06-26 cs.CL 83%

Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models

Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang, Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu

机构 * Tencent AI Lab（腾讯AI实验室）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments ACL 2025

2506.11440 2025-06-16 cs.CL 83%

AbsenceBench: Language Models Can't Tell What's Missing

Harvey Yiyun Fu, Aryan Shrivastava, Jared Moore, Peter West, Chenhao Tan, Ari Holtzman

机构 * University of Chicago（芝加哥大学）； Stanford University（斯坦福大学）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments 23 pages, 8 figures. Code and data are publicly available at https://github.com/harvey-fin/absence-bench

2402.16617 2025-06-11 cs.CL 83%

Long-Context Language Modeling with Parallel Context Encoding

Howard Yen, Tianyu Gao, Danqi Chen

机构 * Princeton Language and Intelligence (PLI)（普林斯顿语言与智能中心）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments ACL 2024. Code, models, and data are available at https://github.com/princeton-nlp/CEPE

2506.06316 2025-06-10 cs.IR cs.AI 83%

A Reinforcement-Learning-Enhanced LLM Framework for Automated A/B Testing in Personalized Marketing

Haoyang Feng, Yanjun Dai, Yuan Gao

专题命中长上下文与记忆：LLM(title,abstract)；language model(abstract)；分类 cs.AI

2506.03090 2025-06-04 cs.CL 83%

Literary Evidence Retrieval via Long-Context Language Models

Katherine Thai, Mohit Iyyer

机构 * UMass Amherst（马萨诸塞大学阿姆赫斯特分校）； University of Maryland, College Park（马里兰大学学院公园分校）

专题命中长上下文与记忆：language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments ACL 2025

2505.13844 2025-05-21 cs.CL 83%

Improve Language Model and Brain Alignment via Associative Memory

Congchi Yin, Yongpeng Zhang, Xuyun Wen, Piji Li

机构 * College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（人工智能学院，南京航空航天大学）； The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education（脑机智能技术重点实验室，教育部）

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted by Findings of ACL 2025

2504.15544 2025-04-23 cs.CL 83%

llm-jp-modernbert: A ModernBERT Model Trained on a Large-Scale Japanese Corpus with Long Context Length

Issa Sugiura, Kouta Nakayama, Yusuke Oda

机构 * Kyoto University（京都大学）； NII LLMC（日本信息处理学会大语言模型中心）

专题命中长上下文与记忆：LLM(title,abstract)；pretraining(abstract)；分类 cs.CL

Comments 9 pages, 5 figures

2503.23053 2025-04-01 cs.CL 83%

A Training-free LLM Framework with Interaction between Contextually Related Subtasks in Solving Complex Tasks

Hongjia Liu, Jinlong Li

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2503.09202 2025-03-13 cs.CL 83%

Token Weighting for Long-Range Language Modeling

Falko Helm, Nico Daheim, Iryna Gurevych

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

Comments Accepted to NAACL 2025 (Findings). For the code, see https://github.com/UKPLab/naacl2025-token-weighting

2503.00353 2025-03-04 cs.CL cs.IR 83%

U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack

Yunfan Gao, Yun Xiong, Wenlong Wu, Zijing Huang, Bohan Li, Haofen Wang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2312.17296 2025-03-03 cs.CL 83%

Structured Packing in LLM Training Improves Long Context Utilization

Konrad Staniszewski, Szymon Tworkowski, Sebastian Jaszczur, Yu Zhao, Henryk Michalewski, Łukasz Kuciński, Piotr Miłoś

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments AAAI'25

2502.20082 2025-02-28 cs.CL 83%

LongRoPE2: Near-Lossless LLM Context Window Scaling

Ning Shang, Li Lyna Zhang, Siyuan Wang, Gaokai Zhang, Gilsinia Lopez, Fan Yang, Weizhu Chen, Mao Yang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2502.00792 2025-02-04 cs.AI 83%

RTBAgent: A LLM-based Agent System for Real-Time Bidding

Leng Cai, Junxuan He, Yikai Li, Junjie Liang, Yuanping Lin, Ziming Quan, Yawen Zeng, Jin Xu

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments Accepted by WWW 2025

2501.16899 2025-01-29 cs.RO cs.AI 83%

RDMM: Fine-Tuned LLM Models for On-Device Robotic Decision Making with Enhanced Contextual Awareness in Specific Domains

Shady Nasrat, Myungsu Kim, Seonil Lee, Jiho Lee, Yeoncheol Jang, Seung-joon Yi

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

2501.03458 2025-01-08 eess.IV cs.AI cs.CV 83%

Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation

Xiao Wang, Fuling Wang, Haowen Wang, Bo Jiang, Chuanfu Li, Yaowei Wang, Yonghong Tian, Jin Tang

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.AI

Comments In Peer Review

2412.10079 2024-12-16 cs.CL 83%

Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA

George Arthur Baker, Ankush Raut, Sagi Shaier, Lawrence E Hunter, Katharina von der Wense

专题命中长上下文与记忆：language model(title,abstract)；prompting(abstract)；分类 cs.CL

2408.10188 2024-12-16 cs.CV cs.CL 83%

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song Han

专题命中长上下文与记忆：language model(title,abstract)；foundation model(abstract)；分类 cs.CL

Comments Code and models are available at https://github.com/NVlabs/VILA/tree/main/longvila

2411.05928 2024-11-12 cs.CL 83%

Reducing Distraction in Long-Context Language Models by Focused Learning

Zijun Wu, Bingyuan Liu, Ran Yan, Lei Chen, Thomas Delteil

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2402.18216 2024-10-15 cs.CL 83%

LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History

Akash Gupta, Ivaxi Sheth, Vyas Raina, Mark Gales, Mario Fritz

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

Comments 20 pages, 13 figures, 20 tables, EMNLP Main Conference 2024

2410.00344 2024-10-08 cs.SD cs.LG eess.AS 83%

Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Pieces

Lilac Atassi

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.LG

Comments arXiv admin note: substantial text overlap with arXiv:2404.11976

2406.14673 2024-10-08 cs.CL 83%

Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell

Taiming Lu, Muhan Gao, Kuai Yu, Adam Byerly, Daniel Khashabi

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2409.09362 2024-09-17 cs.CL 83%

Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM

Yuanjie Lyu, Tong Xu, Zihan Niu, Bo Peng, Jing Ke, Enhong Chen

专题命中长上下文与记忆：LLM(title)；large language model(abstract)；language model(abstract)；分类 cs.CL

2408.07702 2024-08-20 cs.CL 83%

The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models

Karime Maamari, Fadhil Abubaker, Daniel Jaroslawicz, Amine Mhedhbi

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

2407.04923 2024-07-09 cs.CV cs.CL 83%

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding

Tiancheng Zhao, Qianqian Zhang, Kyusong Lee, Peng Liu, Lu Zhang, Chunxin Fang, Jiajia Liao, Kelei Jiang, Yibo Ma, Ruochen Xu

专题命中长上下文与记忆：language model(title)；pretraining(abstract)；prompting(abstract)；分类 cs.CL

Comments 14 pages

2407.03320 2024-07-04 cs.CV cs.CL 83%

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang, Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, Jifeng Dai, Yu Qiao, Dahua Lin, Jiaqi Wang

专题命中长上下文与记忆：language model(title,abstract)；LLM(abstract)；分类 cs.CL

Comments Technical Report. https://github.com/InternLM/InternLM-XComposer

2406.11238 2024-06-18 cs.CL 83%

What Kinds of Tokens Benefit from Distant Text? An Analysis on Long Context Language Modeling

Yutong Hu, Quzhe Huang, Kangcheng Luo, Yansong Feng

专题命中长上下文与记忆：language model(title,abstract)；large language model(abstract)；分类 cs.CL

1. 长上下文与记忆 4822 篇

HoPE: Hybrid of Position Embedding for Long Context Vision-Language Models

No Tokens Wasted: Leveraging Long Context in Biomedical Vision-Language Models

TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialogues

JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model

Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models

AbsenceBench: Language Models Can't Tell What's Missing

Long-Context Language Modeling with Parallel Context Encoding

A Reinforcement-Learning-Enhanced LLM Framework for Automated A/B Testing in Personalized Marketing

Literary Evidence Retrieval via Long-Context Language Models

Improve Language Model and Brain Alignment via Associative Memory

llm-jp-modernbert: A ModernBERT Model Trained on a Large-Scale Japanese Corpus with Long Context Length

A Training-free LLM Framework with Interaction between Contextually Related Subtasks in Solving Complex Tasks

Token Weighting for Long-Range Language Modeling

U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack

Structured Packing in LLM Training Improves Long Context Utilization

LongRoPE2: Near-Lossless LLM Context Window Scaling

RTBAgent: A LLM-based Agent System for Real-Time Bidding

RDMM: Fine-Tuned LLM Models for On-Device Robotic Decision Making with Enhanced Contextual Awareness in Specific Domains

Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation

Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA

LongVILA: Scaling Long-Context Visual Language Models for Long Videos

Reducing Distraction in Long-Context Language Models by Focused Learning

LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History

Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Pieces

Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell

Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM

The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models

OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

What Kinds of Tokens Benefit from Distant Text? An Analysis on Long Context Language Modeling