GRID：持续提示调优中任务无关推理的规模化
GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning
AI总结：
提出GRID框架，通过输出空间感知解码和梯度引导提示选择，解决持续学习中任务无关推理的性能退化与可扩展性问题，在长序列和负迁移基准上提升后向迁移并减少提示内存。
摘要：
基于提示的持续学习提供了一种参数高效的方式，使大型语言模型能够适应任务序列。然而，现有方法通常依赖任务感知推理，并维护不断扩展的任务特定提示集，导致（1）当推理时任务标识符不可用于提示选择时，早期任务性能严重下降；（2）随着任务序列增长，可扩展性受限。我们提出GRID，一个统一的框架来解决这些挑战。GRID包含一个输出空间感知解码机制，通过利用代表性输入和自动标签语义归一化来增强后向迁移，以及一个梯度引导的提示选择策略，将信息量较少的提示压缩为单个聚合表示，以实现可扩展、内存高效的持续学习。在长序列和负迁移基准上的大量实验表明，GRID改善了后向迁移，实现了有竞争力的前向迁移，并显著减少了编码器-解码器和仅解码器架构（包括T5、Qwen和LLaMA）中的提示内存。源代码可从此https URL获取。
英文摘要：
Prompt-based continual learning (CL) offers a parameter-efficient way to adapt large language models (LLMs) across task sequences. However, existing methods often rely on task-aware inference and maintain an expanding set of task-specific prompts, leading to (1) severe performance degradation on earlier tasks when task identifiers are unavailable for prompt selection at inference time, and (2) limited scalability as task sequence grows. We propose GRID, a unified framework designed to address these challenges. GRID incorporates an output-space-aware decoding mechanism that enhances backward transfer by leveraging representative inputs and automatic label semantic normalization, alongside a gradient-guided prompt selection strategy that compresses less informative prompts into a single aggregated representation for scalable, memory-efficient continual learning. Extensive experiments on long-sequence and negative-transfer benchmarks show that GRID improves backward transfer, achieves competitive forward transfer, and substantially reduces prompt memory across encoder-decoder and decoder-only architectures, including T5, Qwen, and LLaMA. Source code is available at https://github.com/AnushkaTi/GRID.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
