DynaMesh：动态3D纹理生成
DynaMesh: Dynamic 3D Texture Generation
浏览论文内容
中文总结 AI 辅助
DynaMesh提出一种动态纹理生成方法，通过视频模型和图像到3D生成器结合时间窗口混合与低秩适配器，在保持几何不变的同时生成随时间演变的纹理，显著优于现有方法。
中文摘要 AI 辅助
我们提出了DynaMesh，一种针对3D网格的动态纹理生成方法。给定一个无纹理的形状和一个描述效果的文本提示，我们的方法生成的外观会随时间演变，而对象的几何形状保持不变。以往关于动态3D内容生成的工作侧重于运动，即对象的几何形状和位置发生变化，而外观保持不变。纹理生成方法则处于问题的另一侧，将外观作为固定的表面属性绘制到形状上，而非一个演变的过程。两者均未处理在3D对象上传播的视觉效果。一种自然的途径包括两个生成器：一个视频模型从单一视角展示效果，以及一个图像到3D的生成器将每一帧提升到3D。然而，后者没有时间概念，因此对每个视频帧运行它会产生闪烁、丢失效果细节且每帧产生不同网格的序列。我们的方法通过将视频模型以网格的渲染和提示为条件来获得参考视频，然后对视频运行冻结的图像到3D生成器，并做了两项改动来解决这些失败。每一帧的条件在时间窗口内进行混合，并为每个形状拟合低秩适配器以恢复丢失的细节。网格对整个序列仅编码一次，因此几何形状在构造上保持不变，输出是带有每帧纹理的单一网格。应用于各种对象和效果，DynaMesh显著优于最近的视频到4D和纹理化方法，并能将其时间效果泛化到训练中从未见过的不同形状。我们的项目页面位于此https URL。
英文摘要
We present DynaMesh, a dynamic texture generation method for 3D meshes. Given a textureless shape and a text prompt describing an effect, our method produces an appearance that evolves while the object's geometry remains unchanged. Previous works on dynamic 3D content generation have focused on motion, where an object's geometry and position change while keeping its appearance the same. Methods on texture generation sit on the other side of the problem, painting appearance onto a shape as a fixed surface property and not as an evolving process. Neither addresses a visual effect that propagates on a 3D object. A natural route consists of two generators: a video model that shows the effect from a single view, and an image-to-3D generator that lifts each frame to 3D. However, the latter has no notion of time, so running it per video frame produces a sequence that flickers, loses effect details, and yields a different mesh at every video frame. Our method addresses these failures by conditioning a video model on a render of the mesh and the prompt to obtain a reference video, then running a frozen image-to-3D generator on the video with two changes. The conditioning of each frame is blended over a temporal window, and low-rank adapters are fit per shape to restore the lost details. The mesh is encoded once for the whole sequence, so geometry is constant by construction, and the output is a single mesh with a texture per frame. Applied to various objects and effects, DynaMesh substantially improves over recent video-to-4D and texturing methods, and can generalize its temporal effect to different shapes never seen during training. Our project page is at https://threedle.github.io/dynamesh/.
发表机构
- University of Chicago（芝加哥大学）
机构由 AI 辅助整理，请以论文原文为准。
