通过视觉上下文学习实现个性化视觉
Personalized Vision via Visual In-Context Learning
AI总结：
本文提出 PICO 四面板框架，利用 diffusion transformers 和 VisRel 数据集实现单示例视觉上下文学习，无需微调即可泛化到用户自定义的识别与生成任务。
摘要：
在大规模标注数据集上训练的现代视觉模型擅长预定义任务，但在个性化视觉方面存在困难——这类任务由用户在测试时用定制对象或新颖目标来定义。现有的个性化方法依赖成本高昂的微调或合成数据流水线，缺乏灵活性，并且局限于固定任务格式。视觉上下文学习（ICL）提供了一种有前景的替代方案，但以往方法仅限于狭窄的域内任务，无法泛化到开放式个性化。我们提出 Personalized In-Context Operator（PICO），这是一个简单的四面板框架，可将 diffusion transformers 重新用作视觉上下文学习者。给定单个带标注示例，PICO 会推断潜在变换，并在无需重新训练的情况下将其应用于新输入。为实现这一点，我们构建了 VisRel，这是一个紧凑而多样的调优数据集，表明驱动鲁棒泛化的是任务多样性而非规模。我们进一步提出注意力引导的种子评分器，通过高效推理缩放提升可靠性。大量实验表明，PICO：（i）优于微调和合成数据基线，（ii）可灵活适应新颖的用户定义任务，并且（iii）能够在识别和生成两类任务间泛化。
英文摘要：
Modern vision models, trained on large-scale annotated datasets, excel at predefined tasks but struggle with personalized vision -- tasks defined at test time by users with customized objects or novel objectives. Existing personalization approaches rely on costly fine-tuning or synthetic data pipelines, which are inflexible and restricted to fixed task formats. Visual in-context learning (ICL) offers a promising alternative, yet prior methods confine to narrow, in-domain tasks and fail to generalize to open-ended personalization. We introduce Personalized In-Context Operator (PICO), a simple four-panel framework that repurposes diffusion transformers as visual in-context learners. Given a single annotated exemplar, PICO infers the underlying transformation and applies it to new inputs without retraining. To enable this, we construct VisRel, a compact yet diverse tuning dataset, showing that task diversity, rather than scale, drives robust generalization. We further propose an attention-guided seed scorer that improves reliability via efficient inference scaling. Extensive experiments demonstrate that PICO (i) surpasses fine-tuning and synthetic-data baselines, (ii) flexibly adapts to novel user-defined tasks, and (iii) generalizes across both recognition and generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
