$ϕ$-DPO：面向大多模态模型持续学习的公平性直接偏好优化方法
$ϕ$-DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Models
AI总结：
本文提出$ϕ$-DPO框架，通过直接偏好优化缓解持续学习中的灾难性遗忘问题，并解决数据不平衡导致的公平性问题，实验表明其在多个基准上表现优异。
摘要：
在大规模多模态模型（LMMs）的持续学习中，公平性是一个新兴但尚未充分研究的挑战，特别是当数据分布不平衡时，可能导致模型更新偏斜和任务性能低下。本文提出一种新的公平性直接偏好优化（FaiDPO或$ϕ$-DPO）框架。首先，我们基于直接偏好优化（DPO）提出新的持续学习范式，通过对齐学习与成对偏好信号来缓解灾难性遗忘。然后，我们识别传统DPO在不平衡数据中的局限性，并提出新的$ϕ$-DPO损失函数，明确解决分布偏置问题。我们提供了全面的理论分析，证明该方法同时解决了遗忘和数据不平衡问题。此外，为了使$ϕ$-DPO支持持续学习，我们为现有基准构建了成对偏好标注。大量实验和消融研究显示，所提出的$ϕ$-DPO在多个基准上实现了最先进的性能，优于先前的LMMs持续学习方法。
英文摘要：
Fairness in Continual Learning for Large Multimodal Models (LMMs) is an emerging yet underexplored challenge, particularly in the presence of imbalanced data distributions that can lead to biased model updates and suboptimal performance across tasks. While recent continual learning studies have made progress in addressing catastrophic forgetting, the problem of fairness caused the imbalanced data remains largely underexplored. This paper presents a novel Fairness Direct Preference Optimization (FaiDPO or $ϕ$-DPO) framework for continual learning in LMMs. In particular, we first propose a new continual learning paradigm based on Direct Preference Optimization (DPO) to mitigate catastrophic forgetting by aligning learning with pairwise preference signals. Then, we identify the limitations of conventional DPO in imbalanced data and present a new $ϕ$-DPO loss that explicitly addresses distributional biases. We provide a comprehensive theoretical analysis demonstrating that our approach addresses both forgetting and data imbalance. Additionally, to enable $ϕ$-DPO-based continual learning, we construct pairwise preference annotations for existing benchmarks in the context of continual learning. Extensive experiments and ablation studies show the proposed $ϕ$-DPO achieves State-of-the-Art performance across multiple benchmarks, outperforming prior continual learning methods of LMMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
