首页/论文/2012.02224

arXiv 2012.022242020-11-11cs.CVcs.GR

Personality-Driven Gaze Animation with Conditional Generative Adversarial Networks

University of Massachusetts Boston（马萨诸塞大学波士顿分校）

Funda Durupinar

备注：: 7 pages, 5 figures

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

We present a generative adversarial learning approach to synthesize gaze behavior of a given personality. We train the model using an existing data set that comprises eye-tracking data and personality traits of 42 participants performing an everyday task. Given the values of Big-Five personality traits (openness, conscientiousness, extroversion, agreeableness, and neuroticism), our model generates time series data consisting of gaze target, blinking times, and pupil dimensions. We use the generated data to synthesize the gaze motion of virtual agents on a game engine.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Unsupervised High-Resolution Portrait Gaze Correction and Animation

University of Trento（特伦托大学）；Zhejiang University（浙江大学）；Shandong University（山东大学）；ETH Zurich（苏黎世联邦理工学院）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；Illinois Institute of Technology（伊利诺伊理工学院）

This paper proposes a gaze correction and animation method for high-resolution, unconstrained portrait images, which can be trained without the gaze angle and the head pose annotations. Common gaze-correction methods usually require annotating training data with precise gaze, and head pose information. Solving this problem using an unsupervised method remains an open problem, especially for high-resolution face images in the wild, which are not easy to annotate with gaze and head pose labels. To address this issue, we first create two new portrait datasets: CelebGaze and high-resolution CelebH…

2022-07-01cs.CVcs.GR

GoHD：具备节律性姿态与逼真表情的面向注视且高度解耦的人像动画生成

GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expression

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；NIO（蔚来）；Banma（斑马）；Beihang University（北京航空航天大学）

针对音频驱动说话人生成的挑战，提出GoHD框架，通过潜空间导航动画模块、conformer条件扩散模型和两阶段训练策略，实现高解耦、表情逼真且泛化性强的可控人像动画生成。

2024-12-13cs.CV

DiffGaze：一种用于在360°图像上生成连续注视序列的扩散模型

DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360° Images

Institute for Visualisation and Interactive Systems, University of Stuttgart（斯图加特大学可视化与交互系统研究所）；KU Leuven（鲁汶大学）

提出DiffGaze，一种基于条件分数去噪扩散模型的新方法，利用360°图像特征和两个transformer建模时空依赖性，生成连续人类注视序列，在多项基准任务上超越现有方法且生成的序列与真实人类序列难以区分。

2024-03-26cs.CVcs.HC

Dual In-painting Model for Unsupervised Gaze Correction and Animation in the Wild

University of Trento（特伦托大学）；Shandong University（山东大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Texas State University（德克萨斯州立大学）；Huawei Research Ireland（华为爱尔兰研究院）

In this paper we address the problem of unsupervised gaze correction in the wild, presenting a solution that works without the need for precise annotations of the gaze angle and the head pose. We have created a new dataset called CelebAGaze, which consists of two domains X, Y, where the eyes are either staring at the camera or somewhere else. Our method consists of three novel modules: the Gaze Correction module (GCM), the Gaze Animation module (GAM), and the Pretrained Autoencoder module (PAM). Specifically, GCM and GAM separately train a dual in-painting network using data from the domain $X…

2020-08-09cs.CVcs.GReess.IV

Negative Shanshui：实时交互式水墨画合成

Negative Shanshui: Real-time Interactive Ink Painting Synthesis

The Hong Kong University of Science and Technology (Guangzhou)（香港理工大学（广州））

本研究提出实时交互式AI合成方法Negative Shanshui，优化微调后的Stable Diffusion模型并结合视线驱动交互等技术，打造VR水墨画体验，经公展反馈验证可引发观众多元情感与思考。

2025-10-05cs.AIcs.CVcs.CY

TalkingEyes: 多元化的语音驱动3D眼动动画

TalkingEyes: Pluralistic Speech-Driven 3D Eye Gaze Animation

School of Informatics, Xiamen University（厦门大学信息学院）；China Mobile (Hangzhou) Information Technology Co., Ltd.（中国移动（杭州）信息技术有限公司）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）

本文提出TalkingEyes，通过语音驱动生成多样化且自然的3D眼动动画，结合头部和面部运动，解决语音与眼动弱相关性问题。

2026-01-04cs.CV

更多相关论文收起更多论文

DiffTalk：为泛化的音频驱动肖像动画构建扩散模型

DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation

Tsinghua University（清华大学）；PhiGent Robotics（PhiGent机器人公司）

本文提出 DiffTalk，将音频驱动说话人头部生成建模为时间一致的潜在扩散去噪过程，并结合参考图像与 landmarks 条件，实现免微调的跨身份高质量、高分辨率同步动画。

2023-04-20cs.CV

Audio- and Gaze-driven Facial Animation of Codec Avatars

Facebook Reality Labs（Facebook现实实验室）；University of Bonn（波恩大学）

Codec Avatars are a recent class of learned, photorealistic face models that accurately represent the geometry and texture of a person in 3D (i.e., for virtual reality), and are almost indistinguishable from video. In this paper we describe the first approach to animate these parametric models in real-time which could be deployed on commodity virtual reality hardware using audio and/or eye tracking. Our goal is to display expressive conversations between individuals that exhibit important social signals such as laughter and excitement solely from latent cues in our lossy input signals. To this…

2020-08-11cs.CV

运动迁移网络改变人类视频中的人格感知

Personality Perception in Human Videos Altered by Motion Transfer Networks

Bilkent University（比尔肯大学）

本研究通过用户研究标注五因素人格样本，用薄板样条运动模型修改视频并设置五组实验，发现运动影响外倾性感知、外貌影响宜人性与神经质感知，明确了动作与外貌对视频人格感知的不同作用。

2024-01-26cs.CV

面向面对面场景的情感检测：眼动时间动态与人格的交互建模

Modelling the Interplay of Eye-Tracking Temporal Dynamics and Personality for Emotion Detection in Face-to-Face Settings

brAIn Lab, IT University of Copenhagen（brAIn实验室，丹麦技术大学）；GN Advanced Science（GN先进科学）；IT University of Copenhagen（丹麦技术大学）

针对动态场景情绪识别难题，提出融合眼动时序动态、大五人格与刺激线索的人格感知多模态框架，在CREMA-D数据集上验证了不同信息对感知/感受情绪的差异化提升作用，推动个性化情感计算发展。

2025-09-19cs.AIcs.CVcs.HC

FaceVR: Real-Time Facial Reenactment and Eye Gaze Control in Virtual Reality

Technical University Munich（慕尼黑工业大学）；Stanford University（斯坦福大学）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；Max-Planck-Institute for Informatics（马克斯·普朗克信息学研究所）

We propose FaceVR, a novel image-based method that enables video teleconferencing in VR based on self-reenactment. State-of-the-art face tracking methods in the VR context are focused on the animation of rigged 3d avatars. While they achieve good tracking performance the results look cartoonish and not real. In contrast to these model-based approaches, FaceVR enables VR teleconferencing using an image-based technique that results in nearly photo-realistic outputs. The key component of FaceVR is a robust algorithm to perform real-time facial motion capture of an actor who is wearing a head-moun…

2018-03-21cs.CV