Connect Later：利用针对性增强提升微调的鲁棒性
Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations
AI总结：
针对预训练后标准微调无法持续降低分布外误差的问题，提出Connect Later方法，预训练用通用增强、微调用针对性增强，在4个真实数据集上取得性能提升。
摘要：
在有标注源域（例如来自野生动物相机陷阱的标注图像）上训练的模型，部署到分布外（OOD）目标域（例如来自新相机陷阱位置的图像）时，泛化性能通常较差。在存在无标注目标数据的域自适应场景中，自监督预训练（例如掩码自编码或对比学习）是缓解这种性能下降的一种有前景的方法。当所使用的通用数据增强（例如掩码或裁剪）能够连接在输入空间中可能相距甚远的源域和目标域时，预训练能够降低OOD误差。本文在真实世界任务中表明，预训练后的标准微调，相较于直接在有标注源数据上从头训练，并不能持续降低OOD误差。为了更好地利用预训练应对分布偏移，我们提出了Connect Later方法：在使用通用增强进行预训练之后，利用结合分布偏移知识设计的针对性增强进行微调。预训练在源域和目标域内学习到良好的表示，而针对性增强在微调过程中更好地连接了两个域。在4个真实世界数据集上，Connect Later相较于标准微调和使用针对性增强的监督学习，降低了平均OOD误差：Connect Later在天文时间序列分类（AstroClassification）任务上取得了当前最优结果，性能提升2.5%；在使用ResNet-50的野生物种识别（iWildCam-WILDS）任务上提升0.9%；在使用DenseNet121的肿瘤识别（Camelyon17-WILDS）任务上提升1.1%；此外在新的天文时间序列红移预测数据集（Redshifts）上取得了最优性能，RMSE降低0.03（相对提升11%）。代码和数据集可在https://github.com/helenqu/connect-later获取。
英文摘要：
Models trained on a labeled source domain (e.g., labeled images from wildlife camera traps) often generalize poorly when deployed on an out-of-distribution (OOD) target domain (e.g., images from new camera trap locations). In the domain adaptation setting where unlabeled target data is available, self-supervised pretraining (e.g., masked autoencoding or contrastive learning) is a promising method to mitigate this performance drop. Pretraining improves OOD error when the generic data augmentations used (e.g., masking or cropping) connect the source and target domains, which may be far apart in the input space. In this paper, we show on real-world tasks that standard fine-tuning after pretraining does not consistently improve OOD error over simply training from scratch on labeled source data. To better leverage pretraining for distribution shifts, we propose Connect Later: after pretraining with generic augmentations, fine-tune with targeted augmentations designed with knowledge of the distribution shift. Pretraining learns good representations within the source and target domains, while targeted augmentations connect the domains better during fine-tuning. Connect Later improves average OOD error over standard fine-tuning and supervised learning with targeted augmentations on 4 real-world datasets: Connect Later achieves the state-of-the-art on astronomical time-series classification (AstroClassification) by 2.5%, wildlife species identification (iWildCam-WILDS) with ResNet-50 by 0.9%, and tumor identification (Camelyon17-WILDS) with DenseNet121 by 1.1%; as well as best performance on a new dataset for astronomical time-series redshift prediction (Redshifts) by 0.03 RMSE (11% relative). Code and datasets are available at https://github.com/helenqu/connect-later.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
