PromptCD：通过极性提示对比解码提升测试时行为
PromptCD: Test-Time Behavior Enhancement via Polarity-Prompt Contrastive Decoding
AI总结：
PromptCD通过极性提示对比解码在测试时提升LLM和VLM的行为一致性，实现无需额外训练的广泛增强。
摘要：
可靠的AI系统需要大语言模型（LLMs）表现出与人类偏好和价值观一致的行为。然而，大多数现有的对齐方法在训练时间进行操作，并依赖于高质量的额外数据，导致显著的计算和标注成本。尽管最近的研究表明，对比解码可以利用模型内部分布来提高特定能力，但其适用性仍然局限于狭窄的行为范围和场景。在本工作中，我们引入了极性提示对比解码（PromptCD），一种在测试时间行为控制的方法，将对比解码扩展到更广泛的增强设置。PromptCD为目标行为构建成对的正负引导提示，并对比模型响应——具体而言是LLM中的token级概率分布和VLM中的视觉注意力模式——以增强期望的结果。这种形式将对比解码扩展到广泛增强目标，并且适用于LLM和视觉语言模型（VLM）而无需额外训练。对于LLM，实验在“3H”对齐目标（帮助性、诚实性和无害性）上显示了一致且显著的改进，表明经过训练的模型可以在测试时间实现有意义的自我增强。对于VLM，我们进一步分析了对比效果对视觉注意力的影响，显示PromptCD通过强化行为一致的视觉基础来显著提高VQA性能。总体而言，这些结果突显PromptCD作为一种简单、通用且成本效益高的策略，适用于跨模态的可靠行为控制。
英文摘要：
Reliable AI systems require large language models (LLMs) to exhibit behaviors aligned with human preferences and values. However, most existing alignment approaches operate at training time and rely on additional high-quality data, incurring significant computational and annotation costs. While recent work has shown that contrastive decoding can leverage a model's internal distributions to improve specific capabilities, its applicability remains limited to narrow behavioral scopes and scenarios. In this work, we introduce Polarity-Prompt Contrastive Decoding (PromptCD), a test-time behavior control method that generalizes contrastive decoding to broader enhancement settings. PromptCD constructs paired positive and negative guiding prompts for a target behavior and contrasts model responses-specifically token-level probability distributions in LLMs and visual attention patterns in VLMs-to reinforce desirable outcomes. This formulation extends contrastive decoding to a wide range of enhancement objectives and is applicable to both LLMs and Vision-Language Models (VLMs) without additional training. For LLMs, experiments on the "3H" alignment objectives (helpfulness, honesty, and harmlessness) demonstrate consistent and substantial improvements, indicating that post-trained models can achieve meaningful self-enhancement purely at test time. For VLMs, we further analyze contrastive effects on visual attention, showing that PromptCD significantly improves VQA performance by reinforcing behavior-consistent visual grounding. Collectively, these results highlight PromptCD as a simple, general, and cost-efficient strategy for reliable behavior control across modalities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
