Tempora: 表征在线测试时适应的时间条件效用
Tempora: Characterising the Time-Contingent Utility of Online Test-Time Adaptation
AI总结：
提出Tempora框架，通过时间场景、评估协议和时间条件效用指标，系统评估测试时适应方法在延迟约束下的准确性-延迟权衡，揭示传统排名在时间压力下失效。
摘要：
测试时适应（TTA）为在域偏移下性能下降的机器学习模型提供了一种引人注目的补救措施，仅使用未标记样本即可即时改进泛化能力。这种灵活性适合实际部署，但传统评估不切实际地假设无限处理时间，忽略了准确性-延迟权衡。随着机器学习越来越多地支撑延迟敏感和面向用户的应用，时间压力限制了可适应推理的可行性；到达太晚而无法采取行动的预测是徒劳的。我们引入了Tempora，一个在这种压力下评估TTA的框架。它由模拟部署约束的时间场景、实现测量的评估协议以及量化准确性-延迟权衡的时间条件效用指标组成。我们用三个这样的指标实例化该框架：（1）用于具有硬截止时间的异步流的离散效用，（2）用于价值随延迟衰减的交互式设置的连续效用，以及（3）用于预算受限部署的摊销效用。通过将Tempora应用于11种TTA方法，我们发现排名不稳定性在跨越不同数据集、模型和硬件平台的750多次时间评估中持续存在；即，传统排名不能预测时间压力下的排名。最高效用方法随偏移和时间压力而变化，没有明确的赢家。通过首次实现跨不同时间约束的系统评估，Tempora揭示了排名何时以及为何变化，为从业者提供了方法选择的视角，为研究人员提供了可部署适应的目标。代码：https://github.com/sudotensor/tempora。
英文摘要：
Test-time adaptation (TTA) offers a compelling remedy for machine learning (ML) models that degrade under domain shifts, improving generalisation on-the-fly with only unlabelled samples. This flexibility suits real deployments, yet conventional evaluations unrealistically assume unbounded processing time, overlooking the accuracy-latency trade-off. As ML increasingly underpins latency-sensitive and user-facing use-cases, temporal pressure constrains the viability of adaptable inference; predictions arriving too late to act on are futile. We introduce Tempora, a framework for evaluating TTA under this pressure. It consists of temporal scenarios that model deployment constraints, evaluation protocols that operationalise measurement, and time-contingent utility metrics that quantify the accuracy-latency trade-off. We instantiate the framework with three such metrics: (1) discrete utility for asynchronous streams with hard deadlines, (2) continuous utility for interactive settings where value decays with latency, and (3) amortised utility for budget-constrained deployments. By applying Tempora to 11 TTA methods, we find that rank instability persists across 750+ temporal evaluations spanning diverse datasets, models, and hardware platforms; i.e., conventional rankings do not predict rankings under temporal pressure. The highest-utility method varies with the shift and temporal pressure, with no clear winner. By enabling systematic evaluation across diverse temporal constraints for the first time, Tempora reveals when and why rankings change, offering practitioners a lens for method selection and researchers a target for deployable adaptation. Code: https://github.com/sudotensor/tempora.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
