GRAPE: 可泛化且鲁棒的多视图面部捕捉
GRAPE: Generalizable and Robust Multi-view Facial Capture
AI总结：
针对现有多视图面部捕捉系统泛化性差的问题，提出GRAPE系统，通过相机阵列无关初始化模块和“按分歧更新”策略实现跨相机阵列的鲁棒面部捕捉，在FaMoS和FaceScape数据集上验证有效。
摘要：
基于深度学习的多视图面部捕捉方法已展现出令人瞩目的精度，同时其速度比传统网格配准流程快几个数量级。然而，现有系统（例如TEMPEH）严格受限于只能对用于捕获其训练数据的同一相机阵列所采集的数据进行推理。本研究旨在提升泛化能力，使训练好的模型能够直接用于不同相机阵列上的推理（即捕捉新数据）。为此，我们提出了一个更具泛化性的初始化模块，用于提取与相机阵列无关的3D特征，该模块包含基于视觉外壳的头部定位，以及由视觉外壳实现的感知可见性的3D特征聚合模块。此外，我们提出了一种“按分歧更新”的学习策略，通过在训练期间丢弃潜在不准确的监督信号，来更好地处理数据噪声（例如不准确的配准、扫描噪声）。由此得到的可泛化且鲁棒的拓扑一致多视图面部捕捉系统（GRAPE）可直接用于在不同相机阵列上捕捉数据，大幅减少了数据收集和处理的工作量。在FaMoS和FaceScape数据集上的实验证明了所提方法的有效性。
英文摘要：
Deep learning-based multi-view facial capture methods have shown impressive accuracy while being several orders of magnitude faster than a traditional mesh registration pipeline. However, the existing systems (e.g. TEMPEH) are strictly restricted to inference on the data captured by the same camera array used to capture their training data. In this study, we aim to improve the generalization ability so that a trained model can be readily used for inference (i.e. capture new data) on a different camera array. To this end, we propose a more generalizable initialization module to extract the camera array-agnostic 3D feature, including a visual hull-based head localization and a visibility-aware 3D feature aggregation module enabled by the visual hull. In addition, we propose an ``update-by-disagreement'' learning strategy to better handle data noise (e.g. inaccurate registration, scan noise) by discarding potentially inaccurate supervision signals during training. The resultant generalizable and robust topologically consistent multi-view facial capture system (GRAPE) can be readily used to capture data on a different camera array, reducing great effort on data collection and processing. Experiments on the FaMoS and FaceScape datasets demonstrate the effectiveness of the proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
