面向多模态持续食品学习的双LoRA与质量增强伪回放方法
Dual-LoRA and Quality-Enhanced Pseudo Replay for Multimodal Continual Food Learning
AI总结：
针对多模态食品分析大模型学习新任务时的灾难性遗忘问题，提出融合Dual-LoRA架构与质量增强伪回放的持续学习框架，在Uni-Food数据集上验证了其优异的遗忘缓解效果。
摘要：
食品分析对于个性化营养、慢性病预防等健康相关任务愈发关键，但现有用于食品分析的大多模态模型（LMMs）在学习新任务时会遭遇灾难性遗忘问题，需要付出高昂成本从头重新训练。为解决该问题，本文提出一种新型多模态食品学习持续学习框架，将Dual-LoRA架构与质量增强伪回放机制相结合。我们为每个任务引入两个互补的低秩适配器：一个是专用LoRA，在与先前任务子空间正交的约束下学习任务特定知识；另一个是协作LoRA，通过伪回放整合跨任务的共享知识。为提升回放数据可靠性，我们的质量增强伪回放策略利用自一致性与语义相似度减少生成样本中的幻觉现象。在全面的Uni-Food数据集上开展的实验表明，该方法在缓解遗忘方面表现优异，是首个面向复杂食品任务的有效持续学习方案。
英文摘要：
Food analysis has become increasingly critical for health-related tasks such as personalized nutrition and chronic disease prevention. However, existing large multimodal models (LMMs) in food analysis suffer from catastrophic forgetting when learning new tasks, requiring costly retraining from scratch. To address this, we propose a novel continual learning framework for multimodal food learning, integrating a Dual-LoRA architecture with Quality-Enhanced Pseudo Replay. We introduce two complementary low-rank adapters for each task: a specialized LoRA that learns task-specific knowledge with orthogonal constraints to previous tasks' subspaces, and a cooperative LoRA that consolidates shared knowledge across tasks via pseudo replay. To improve the reliability of replay data, our Quality-Enhanced Pseudo Replay strategy leverages self-consistency and semantic similarity to reduce hallucinations in generated samples. Experiments on the comprehensive Uni-Food dataset show superior performance in mitigating forgetting, representing the first effective continual learning approach for complex food tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
