大型视觉-语言模型的对齐与失齐：基于可解释性视角的综述
Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainability
AI总结：
本文从可解释性视角系统综述了大型视觉-语言模型的对齐与失齐问题，分析对象、属性、关系三层失齐现象及其数据、模型、推理层面的成因，并梳理参数冻结与参数微调两类缓解策略，指出未来需建立标准化评估协议和深化可解释性研究。
摘要：
大型视觉-语言模型（LVLMs）在处理视觉和文本信息方面已展现出卓越能力。然而，视觉表示与文本表示之间对齐这一关键挑战尚未被充分理解。本综述通过可解释性视角，对LVLMs中的对齐与失齐现象进行了全面考察。我们首先审视对齐的基础，探讨其表示层面和行为层面、训练方法以及理论基础。随后，我们在三个语义层级上分析失齐现象：对象失齐、属性失齐和关系失齐。我们的研究表明，失齐源于多个层面的挑战：数据层面、模型层面和推理层面。我们对现有缓解策略进行了全面回顾，将其划分为参数冻结方法和参数微调方法两类。最后，我们概述了有前景的未来研究方向，强调需要标准化的评估协议和深入的可解释性研究。
英文摘要：
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in processing both visual and textual information. However, the critical challenge of alignment between visual and textual representations is not fully understood. This survey presents a comprehensive examination of alignment and misalignment in LVLMs through an explainability lens. We first examine the fundamentals of alignment, exploring its representational and behavioral aspects, training methodologies, and theoretical foundations. We then analyze misalignment phenomena across three semantic levels: object, attribute, and relational misalignment. Our investigation reveals that misalignment emerges from challenges at multiple levels: the data level, the model level, and the inference level. We provide a comprehensive review of existing mitigation strategies, categorizing them into parameter-frozen and parameter-tuning approaches. Finally, we outline promising future research directions, emphasizing the need for standardized evaluation protocols and in-depth explainability studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
