感知历史的交叉注意力强化：基于vLLM的自监督多轮与思维链微调
History-Aware Cross-Attention Reinforcement: Self-Supervised Multi Turn and Chain-of-Thought Fine-Tuning with vLLM
AI总结：
本研究提出在vLLM上实现的CAGSR框架扩展CAGSR-vLLM-MTC，通过改造内核异步捕获注意力权重、泛化奖励函数适配多轮对话与思维链推理，还提出熵钳制机制防注意力崩塌，为相关后续研究提供方向。
摘要：
我们提出CAGSR-vLLM-MTC，它是自监督交叉注意力引导强化（CAGSR）框架的扩展版本，目前已在高性能vLLM运行时上实现，用于解决多轮对话与思维链推理问题。基于原有的单轮方法，我们首先对vLLM的C++/CUDA内核进行改造，在生成过程中异步捕获每层、每个注意力头的交叉注意力权重。随后我们将自监督奖励函数泛化，使其能在整个对话历史和中间思维链步骤上累积注意力信号。我们讨论了实际应用中的权衡，包括一种基于熵的钳制机制，用于防止注意力在早期上下文上崩塌，并概述了多方对话与分层推理的未来研究方向。
英文摘要：
We present CAGSR-vLLM-MTC, an extension of our Self-Supervised Cross-Attention-Guided Reinforcement (CAGSR) framework, now implemented on the high-performance vLLM runtime, to address both multi-turn dialogue and chain-of-thought reasoning. Building upon our original single-turn approach, we first instrumented vLLM's C++/CUDA kernels to asynchronously capture per-layer, per-head cross-attention weights during generation. We then generalized our self-supervised reward function to accumulate attention signals over entire conversation histories and intermediate chain-of-thought steps. We discuss practical trade-offs, including an entropy-based clamping mechanism to prevent attention collapse on early context, and outline future directions for multi-party dialogues and hierarchical reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
