TARDIS:通过表示引导缓解时间错位
TARDIS: Mitigating Temporal Misalignment via Representation Steering
AI总结:
TARDIS是一种无监督表示编辑方法,通过提取引导向量调整模型表示,无需微调即可缓解时间错位,提升下游任务性能,且对数据可用性要求低。
摘要:
语言模型经常面临时间错位问题,即由于数据时间分布的变化而导致的性能下降。持续更新模型以避免性能下降成本高昂。能否在不更新模型权重的情况下使模型适应?我们提出了TARDIS,一种无监督的表示编辑方法,以解决这一挑战。TARDIS从无标签数据中提取引导向量,并调整模型的表示,使其更好地与目标时间段的数据分布对齐。我们的实验表明,TARDIS无需微调即可提升下游任务性能,即使在无法获得精确目标时间段数据的情况下也能缓解时间错位,并且在推理时目标数据点的时间信息未知时仍能保持高效。
英文摘要:
Language models often struggle with temporal misalignment, performance degradation caused by shifts in the temporal distribution of data. Continuously updating models to avoid degradation is expensive. Can models be adapted without updating model weights? We present TARDIS, an unsupervised representation editing method that addresses this challenge. TARDIS extracts steering vectors from unlabeled data and adjusts the model's representations to better align with the target time period's distribution. Our experiments reveal that TARDIS enhances downstream task performance without the need for fine-tuning, can mitigate temporal misalignment even when exact target time period data is unavailable, and remains efficient even when the temporal information of the target data points is unknown at inference time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。