情感图像编辑旨在改变场景所传达的情感,同时保留其核心内容。由于同一场景可以支持不同的情感解读,要实现情感目标需要基于源上下文进行视觉改变。受将情感反应与事件评估关联起来的组件过程模型(Component Process Model)的启发,我们研究神经表征作为这些编辑决策的参考。使用fMRI编码模型TRIBE v2,我们分析了8个情感类别中的400个源-目标对。EmoEdit、EmoEditor和AffectDelta的隐藏状态在视觉区域中与预测的目标图像反应的对应关系,比在情感相关区域中更强。这种不平衡促使我们提出ROI-ReFT,它将源视觉反应作为内容参考,将预测的情感反应变化作为编辑方向。两个交互分支利用这两个条件在整个生成过程中指导一个冻结的编辑器。生成图像的预测反应监督情感变化并约束视觉漂移。在一个事后选择的400个编辑请求子集上,ROI-ReFT实现了91.75%的目标情感准确率和83.25%的联合成功率,其输出既匹配目标情感,又达到源输出DINO-I至少0.80。我们的代码和数据集将公开发布。
英文摘要:
Emotional image editing aims to change the emotion a scene conveys while preserving its essential content. Because the same scene can support different emotional interpretations, achieving an emotional goal requires visual changes grounded in the source context. Motivated by the Component Process Model, which links emotional responses to event appraisal, we investigate neural representations as a reference for these editing decisions. Using TRIBE v2, an fMRI encoding model, we analyze 400 source-target pairs across eight emotion categories. Hidden states from EmoEdit, EmoEditor, and AffectDelta correspond more strongly to predicted target-image responses in visual regions than in affect-related regions. This imbalance motivates ROI-ReFT, which combines source visual responses as a content reference with a predicted affective response change as an editing direction. Two interacting branches use both conditions to guide a frozen editor throughout generation. Predicted responses of the generated image supervise affective change and constrain visual drift. On a post hoc selected subset of 400 editing requests, ROI-ReFT achieves 91.75% target-emotion accuracy and 83.25% joint success, with the same output both matching the target emotion and reaching source-output DINO-I of at least 0.80. Our code and dataset will be publicly released.
Affective Image Editing: Shaping Emotional Factors via Text Descriptions
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China(北京邮电大学人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);State Key Laboratory for Multimedia Information Processing and National Engineering Research Center of Visual Technology, School of Computer Science, Peking University, China(多媒体信息处理国家重点实验室和视觉技术国家工程研究中心,北京大学计算机学院)
AUHead: Realistic Emotional Talking Head Generation via Action Units Control
University of the Chinese Academy of Sciences(中国科学院大学);National University of Singapore(新加坡国立大学);Zhejiang University(浙江大学);State Key Laboratory of Communication Content Cognition, People’s Daily Online(人民日報網通信內容認知重點實驗室)
MUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization
VCIP & TMCC & DISSec, College of Computer Science, Nankai University(南开大学计算机学院);LIX, Ecole Polytechnique, IP Paris(巴黎高等理工学院);Key Lab of SCCI, Dalian University of Technology(大连理工大学SCCI重点实验室)
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Tencent AI Lab(腾讯人工智能实验室);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape
School of Artificial Intelligence, Shenzhen Technology University(深圳技术大学人工智能学院);School of Innovation Design, Shenzhen Technology University(深圳技术大学创新设计学院);Stanford University(斯坦福大学)
Regressor-Guided Image Editing Shifts Emotion and Disengagement Timing in Social Media
Eastern Switzerland University of Applied Sciences \& ETH Z\"urich St.Gallen Switzerland;ETH Z\"urich Z\"urich Switzerland;The Hong Kong University of Science;Eastern Switzerland University of Applied Sciences \& ETH Z\"urich;ETH Z\"urich
MooD: Perception-Enhanced Efficient Affective Image Editing via Continuous Valence-Arousal Modeling
School of Cyber Science and Engineering, Zhengzhou University(郑州大学信息科学与工程学院);School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);School of Journalism and New Media, Xi’an Jiaotong University(西安交通大学新闻与传播学院);Research Center for Space Computing System, Zhejiang Lab(浙江实验室空间计算系统研究中心)
FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing
Xi’an Key Laboratory of Big Data and Intelligent Vision, Xidian University, Xi’an 710071, China(西安大数据与智能视觉重点实验室,西安电子科技大学,西安710071,中国);Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University, Xi’an 710071, China(协同智能系统重点实验室,教育部,西安电子科技大学,西安710071,中国);School of Computer Science and Technology, Xidian University, Xi’an 710071, China(计算机科学与技术学院,西安电子科技大学,西安710071,中国)
EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models
University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology Beijing(北京科技大学);The New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室模式识别新实验室);The New Laboratory of Pattern Recognition (NLPR)(模式识别新实验室)
DPE: Disentanglement of Pose and Expression for General Video Portrait Editing
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Tencent AI Lab(腾讯人工智能实验室)
MAGIC-Talk: Motion-aware Audio-Driven Talking Face Generation with Customizable Identity Control
School of Computer Science and Electronic Engineering, University of Surrey(Surrey大学计算机科学与电子工程学院);School of Artificial Intelligence and Computer Science, Jiangnan University(江南大学人工智能与计算机科学学院);Centre for Vision, Speech and Signal Processing, University of Surrey(Surrey大学视觉、语音和信号处理中心)
Geometry-guided Emotion Modulation for Controllable and Photorealistic Emotional Talking Face Generation
School of Software Technology, Zhejiang University(浙江大学软件学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Ningbo Global Innovation Center, Zhejiang University(浙江大学宁波全球创新中心)