arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.13558cs.CV
收藏

当我们谈论情感编辑时,我们谈论的是什么

What We Talk About When We Talk About Emotion Editing

Vanderbilt University(范德堡大学);Research Institute of Electrical Communication, Tohoku University(东北大学电气通信研究所);RIKEN(理化学研究所)

Xingzu Zhan, Lin Gu, Ruogu Fang

AI总结:

该研究针对情感图像编辑中视觉与情感区域表征不平衡的问题,提出ROI-ReFT方法,结合源视觉参考与预测情感变化,在400个编辑请求子集上取得91.75%目标情感准确率等成果,代码与数据集将公开。

摘要:

情感图像编辑旨在改变场景所传达的情感,同时保留其核心内容。由于同一场景可以支持不同的情感解读,要实现情感目标需要基于源上下文进行视觉改变。受将情感反应与事件评估关联起来的组件过程模型(Component Process Model)的启发,我们研究神经表征作为这些编辑决策的参考。使用fMRI编码模型TRIBE v2,我们分析了8个情感类别中的400个源-目标对。EmoEdit、EmoEditor和AffectDelta的隐藏状态在视觉区域中与预测的目标图像反应的对应关系,比在情感相关区域中更强。这种不平衡促使我们提出ROI-ReFT,它将源视觉反应作为内容参考,将预测的情感反应变化作为编辑方向。两个交互分支利用这两个条件在整个生成过程中指导一个冻结的编辑器。生成图像的预测反应监督情感变化并约束视觉漂移。在一个事后选择的400个编辑请求子集上,ROI-ReFT实现了91.75%的目标情感准确率和83.25%的联合成功率,其输出既匹配目标情感,又达到源输出DINO-I至少0.80。我们的代码和数据集将公开发布。

英文摘要:

Emotional image editing aims to change the emotion a scene conveys while preserving its essential content. Because the same scene can support different emotional interpretations, achieving an emotional goal requires visual changes grounded in the source context. Motivated by the Component Process Model, which links emotional responses to event appraisal, we investigate neural representations as a reference for these editing decisions. Using TRIBE v2, an fMRI encoding model, we analyze 400 source-target pairs across eight emotion categories. Hidden states from EmoEdit, EmoEditor, and AffectDelta correspond more strongly to predicted target-image responses in visual regions than in affect-related regions. This imbalance motivates ROI-ReFT, which combines source visual responses as a content reference with a predicted affective response change as an editing direction. Two interacting branches use both conditions to guide a frozen editor throughout generation. Predicted responses of the generated image supervise affective change and constrain visual drift. On a post hoc selected subset of 400 editing requests, ROI-ReFT achieves 91.75% target-emotion accuracy and 83.25% joint success, with the same output both matching the target emotion and reaching source-output DINO-I of at least 0.80. Our code and dataset will be publicly released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑