读作人类：通过可并行的近读与略读压缩上下文
Read As Human: Compressing Context via Parallelizable Close Reading and Skimming
AI总结：
RAM通过可并行的近读与略读策略，有效压缩上下文，提升长输入处理效率，实现性能与可解释性的双重优化。
摘要：
大型语言模型（LLMs）在各种任务中表现出卓越的能力。然而，在长上下文场景中部署它们受到两个挑战的阻碍：计算效率低下和冗余信息。我们提出了RAM（Read As HuMan），一个上下文压缩框架，采用一种自适应的混合阅读策略，以解决这些问题。受人类阅读行为（即对重要内容进行近读，对不相关内容进行略读）的启发，RAM将上下文分成段落，并与输入查询并行编码。高相关性段落被完全保留（近读），而低相关性段落则通过查询引导压缩成紧凑的摘要向量（略读）。显式文本段落和隐式摘要向量被连接并输入解码器，以实现优越的性能和自然语言格式的可解释性。为了细化近读与略读之间的决策边界，我们进一步引入了一种基于正负查询-段落对的对比学习目标。实验表明，RAM在多个问答和摘要基准测试中优于现有基线，跨两个backbone表现优异，同时在长输入上实现了高达12倍的端到端加速（平均长度16K；最大长度32K）。
英文摘要：
Large Language Models (LLMs) demonstrate exceptional capability across diverse tasks. However, their deployment in long-context scenarios is hindered by two challenges: computational inefficiency and redundant information. We propose RAM (Read As HuMan), a context compression framework that adopts an adaptive hybrid reading strategy, to address these challenges. Inspired by human reading behavior (i.e., close reading important content while skimming less relevant content), RAM partitions the context into segments and encodes them with the input query in parallel. High-relevance segments are fully retained (close reading), while low-relevance ones are query-guided compressed into compact summary vectors (skimming). Both explicit textual segments and implicit summary vectors are concatenated and fed into decoder to achieve both superior performance and natural language format interpretability. To refine the decision boundary between close reading and skimming, we further introduce a contrastive learning objective based on positive and negative query-segment pairs. Experiments demonstrate that RAM outperforms existing baselines on multiple question answering and summarization benchmarks across two backbones, while delivering up to a 12x end-to-end speedup on long inputs (average length 16K; maximum length 32K).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
