We propose 3Deformer, a general-purpose framework for interactive 3D shape editing. Given a source 3D mesh with semantic materials, and a user-specified semantic image, 3Deformer can accurately edit the source mesh following the shape guidance of the semantic image, while preserving the source topology as rigid as possible. Recent studies of 3D shape editing mostly focus on learning neural networks to predict 3D shapes, which requires high-cost 3D training datasets and is limited to handling objects involved in the datasets. Unlike these studies, our 3Deformer is a non-training and common framework, which only requires supervision of readily-available semantic images, and is compatible with editing various objects unlimited by datasets. In 3Deformer, the source mesh is deformed utilizing the differentiable renderer technique, according to the correspondences between semantic images and mesh materials. However, guiding complex 3D shapes with a simple 2D image incurs extra challenges, that is, the deform accuracy, surface smoothness, geometric rigidity, and global synchronization of the edited mesh should be guaranteed. To address these challenges, we propose a hierarchical optimization architecture to balance the global and local shape features, and propose further various strategies and losses to improve properties of accuracy, smoothness, rigidity, and so on. Extensive experiments show that our 3Deformer is able to produce impressive results and reaches the state-of-the-art level.
HeartVolMesh: Cardiac Volumetric Mesh Reconstruction via Covariance-Guided Graph Deformation
Centre for Computational Imaging and Modelling in Medicine (CIMIM), University of Manchester（曼彻斯特大学医学计算成像与建模中心（CIMIM））；Christabel Pankhurst Institute, University of Manchester（曼彻斯特大学克里斯塔贝尔·潘克赫斯特研究所）；Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Division of Informatics, Imaging & Data Sciences, University of Manchester（曼彻斯特大学信息学、成像与数据科学部）；NIHR Manchester Biomedical Research Centre, Manchester Academic Health Sciences Centre, University of Manchester（曼彻斯特大学NIHR曼彻斯特生物医学研究中心，曼彻斯特学术健康科学中心）
3D Hand Mesh-Guided AI-Generated Malformed Hand Refinement with Hand Pose Transformation via Diffusion Model
Department of Computer and Information Science, University of Macau, Macau, China（计算机与信息科学系，澳门大学，澳门，中国）；ShanghaiTech University, Shanghai, China（上海科技大学，上海，中国）
SemMorph3D: Unsupervised Semantic-Aware 3D Morphing via Mesh-Guided Gaussians
Shanghai University School of Artificial Intelligence , Wuhan University Tencent Shanghai Engineering Research Center of Motion Picture Special Effects（上海大学人工智能学院、武汉大学、腾讯、上海影视特效工程研究中心）
TFormer: 3D Tooth Segmentation in Mesh Scans with Geometry Guided Transformer
Zhejiang University（浙江大学）；Angelalign Research Institute（时代天使研究院）；Angel Align Inc.（时代天使公司）；A*STAR Centre for Frontier AI Research (CFAR)（新加坡科技研究局前沿人工智能研究中心）；Stomatology Hospital, School of Stomatology, Zhejiang University School of Medicine（浙江大学医学院附属口腔医院）；Harvard University（哈佛大学）
TokenMatch: 3D Mesh Correspondence Transformer with Curvature-Guided Tokenisation
Italian Institute of Technology（意大利理工学院）；University of Genoa（热那亚大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；University of Bonn（波恩大学）；Lamarr Institute（拉马尔研究所）；University of Verona（维罗纳大学）
GReFEM: Multimodal LLMs as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement
Institute of Material Systems Modeling（材料系统建模研究所）；Helmholtz Zentrum Hereon（海德堡研究中心Hereon）；Institute of Material and Process Design（材料与加工设计研究所）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
University of Liverpool（利物浦大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；Universidad Antonio de Nebrija（安东尼奥·德·内布里哈大学）；ARIES Research Centre（ARIES研究中心）
GeoSplatting: Towards Geometry Guided Gaussian Splatting for Physically-based Inverse Rendering
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）
ProxyBuild: Text-Guided Structured 3D Building Generation with Mesh-Anchored Procedural Proxies
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Pengcheng Laboratory（鹏城实验室）；Harbin Institute of Technology（哈尔滨工业大学）；Harbin Institute of Technology, Suzhou Research Institute（哈尔滨工业大学苏州研究院）
Drive Any Mesh: 4D Latent Diffusion for Mesh Deformation from Video
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（虚拟现实技术与系统国家重点实验室，北京航空航天大学）；School of Electronic and Computer Engineering, Peking University（电子与计算机工程学院，北京大学）；Baidu VIS（百度VIS）
AttenIR: Aligning Dense Visual Attention with Mesh Warping for Unsupervised Image Retargeting
School of Media Engineering, Communication University of Zhejiang（浙江传媒学院媒体工程学院）；Samsung Research America（三星美国研究院）；School of Computer Science and Engineering, The University of New South Wales（新南威尔士大学计算机科学与工程学院）
