OFERA：基于混合形状的3D高斯控制用于VR中被遮挡面部表情到真实avatar
OFERA: Blendshape-driven 3D Gaussian Control for Occluded Facial Expression to Realistic Avatars in VR
AI总结：
OFERA通过混合形状信号实现VR中逼真avatar的实时表情控制，提升远程存在感。
摘要：
我们提出OFERA，一种用于VR头显用户实时表达控制的逼真高斯头avatar框架。现有方法试图使用额外传感器或内部摄像头恢复被遮挡的面部表情，但基于传感器的方法会增加设备重量和不适，而基于摄像头的方法则引发隐私问题并受限于原始数据的访问。为克服这些限制，我们利用商业VR头显提供的混合形状信号作为表情输入。我们的框架由三个关键组件组成：（1）混合形状分布对齐（BDA），它通过线性回归将头显提供的混合形状分布对齐到一个标准输入空间；（2）一个表情参数映射器（EPM），将对齐后的混合形状信号映射到一个表情参数空间以控制高斯头avatar；（3）一个集成了映射器的avatar（MiA），将EPM整合到avatar学习过程中以确保分布一致性。此外，OFERA建立了一个端到端的流程，用于感知和映射表情、更新高斯avatar并实时在VR环境中渲染。我们展示了EPM在定量指标上优于现有映射方法，并通过用户研究证明了完整的OFERA框架在提升表情保真度的同时保持avatar的现实性。通过实现实时和逼真的avatar表情控制，OFERA显著提高了VR通信中的远程存在感。项目页面可在https://ysshwan147.github.io/projects/ofera/上找到。
英文摘要：
We propose OFERA, a novel framework for real-time expression control of photorealistic Gaussian head avatars for VR headset users. Existing approaches attempt to recover occluded facial expressions using additional sensors or internal cameras, but sensor-based methods increase device weight and discomfort, while camera-based methods raise privacy concerns and suffer from limited access to raw data. To overcome these limitations, we leverage the blendshape signals provided by commercial VR headsets as expression inputs. Our framework consists of three key components: (1) Blendshape Distribution Alignment (BDA), which applies linear regression to align the headset-provided blendshape distribution to a canonical input space; (2) an Expression Parameter Mapper (EPM) that maps the aligned blendshape signals into an expression parameter space for controlling Gaussian head avatars; and (3) a Mapper-integrated Avatar (MiA) that incorporates EPM into the avatar learning process to ensure distributional consistency. Furthermore, OFERA establishes an end-to-end pipeline that senses and maps expressions, updates Gaussian avatars, and renders them in real-time within VR environments. We show that EPM outperforms existing mapping methods on quantitative metrics, and we demonstrate through a user study that the full OFERA framework enhances expression fidelity while preserving avatar realism. By enabling real-time and photorealistic avatar expression control, OFERA significantly improves telepresence in VR communication. A project page is available at https://ysshwan147.github.io/projects/ofera/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
