G2LoRA: 面向文本属性图的梯度正交低秩自适应框架用于图持续学习
G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs
AI总结：
针对LLM-as-Aligner模型在文本属性图持续学习中的灾难性遗忘问题，提出G2LoRA框架，通过统一图-文本对齐目标、类别感知梯度投影和梯度幅度调制，实现任务间正向迁移并缓解模态漂移。
摘要：
LLM-as-Aligner已成为文本属性图（TAGs）的一种流行预训练范式，通过CLIP风格的对比学习将图和文本模态对齐到共享嵌入空间。虽然在单个下游任务上有效，但我们观察到当此类模型在流式任务上顺序微调时会出现严重的灾难性遗忘。尽管参数高效微调在一定程度上缓解了遗忘，但仍不足以解决任务干扰和无效知识迁移。在这项工作中，我们研究了TAGs上LLM-as-Aligner模型的图持续学习，目标是减轻干扰同时促进任务间的正向迁移。该设置引入了两个基本挑战：（1）异构下游任务导致优化目标变化，阻碍统一微调；（2）图和文本编码器对自适应表现出不同的敏感性，不协调的更新容易导致错位。为应对这些挑战，我们提出了G2LoRA，一个面向TAGs的持续学习框架。G2LoRA将节点级、链接级和图级任务统一到单一的图-文本对齐目标下，并在领域/类别/任务增量模式下实现一致的优化。为减少任务干扰同时鼓励正向迁移，G2LoRA在结构化子空间中执行类别感知梯度投影，解决冲突更新并实现条件性反向迁移以平衡前向和后向知识流。为进一步防止跨模态漂移，G2LoRA引入梯度幅度调制来协调图和文本编码器之间的更新速率。在基准数据集上的大量实验表明，G2LoRA在不同骨干架构上始终优于强基线，实现了卓越的持续性能和可迁移性。
英文摘要：
LLM-as-Aligner has emerged as a prevalent pre-training paradigm for Text-Attributed Graphs(TAGS), aligning graph and text modalities into a shared embedding space via CLIP-style contrastive learning. While effective on individual downstream tasks, we observe severe catastrophic forgetting when such models are sequentially fine-tuned on streaming tasks. Although parameter-efficient fine-tuning alleviates forgetting to some extent, it remains insufficient to resolve task interference and ineffective knowledge transfer. In this work, we study graph continual learning for LLM-as-Aligner models on TAGs, with the goal of mitigating interference while promoting positive transfer across tasks. This setting introduces two fundamental challenges: (1) heterogeneous downstream tasks induce shifting optimization objectives, hindering unified fine-tuning; and (2) graph and text encoders exhibit different sensitivities to adaptation, making uncoordinated updates prone to misalignment. To address these challenges, we propose G2LoRA, a continual learning framework for TAGs. G2LoRA unifies node-, link-, and graph-level tasks under a single graph--text alignment objective, and enables consistent optimization across domain/class/task incremental modes. To reduce task interference while encouraging positive transfer, G2LoRA performs category-aware gradient projection in structured subspaces, resolving conflicting updates and enabling conditional backward transfer to balance forward and backward knowledge flow. To further prevent cross-modal drift, G2LoRA introduces gradient magnitude modulation to coordinate update rates between graph and text encoders. Extensive experiments on benchmark datasets demonstrate that G2LoRA consistently outperforms strong baselines across different backbone architectures, achieving superior continual performance and transferability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
