发表机构
Institute of Automation, Chinese Academy of Science; University of Chinese Academy of Sciences; Shanghaitech University（中国科学院自动化研究所; 中国科学院大学; 上海科技大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对3D和4D内容生成中的时空幻觉问题，提出Hallo4D框架，利用多模态语言模型，通过生成-检测-校正范式及多种技术提升一致性，实验证明其在多样生成设置下优于基线，提供了可扩展通用的一致性感知内容生成方案。
AI 中文摘要
虽然最近3D生成的进展实现了令人印象深刻的视觉合成，但现有方法通常依赖2D扩散监督，缺乏明确的几何一致性机制，导致空间幻觉。在4D生成中这些问题更严重。我们提出Hallo4D，一个统一且与模型无关的框架，用于缓解3D和4D内容生成中的时空幻觉。它引入生成-检测-校正范式，利用大型多模态语言模型识别和总结多视图和多帧渲染中的时空不一致，通过多模型投票评估候选校正，还采用多种技术提升性能。实验表明Hallo4D在多种设置下优于基线。
英文摘要
While recent advances in 3D generation have enabled impressive visual synthesis, existing methods often rely on 2D diffusion supervision without explicit mechanisms for geometric consistency, leading to spatial hallucinations such as duplicated structures and misaligned geometry. These issues become more severe in 4D generation, where maintaining consistency across viewpoints and temporal evolution introduces additional challenges, including jitter, identity flicker, and structural drift. We present \textbf{Hallo4D}, a unified and model-agnostic framework for mitigating spatiotemporal hallucinations in 3D and 4D content generation. Hallo4D introduces a generation-detection-correction paradigm that leverages large multimodal language models (LMMs) to identify and summarize spatial and temporal inconsistencies from multi-view and multi-frame renderings. These insights guide a consensus-driven image-space consistency optimization, where an LMM-based selector evaluates candidate corrections through multi-model voting, without requiring retraining or architectural modifications. To further improve temporal consistency and optimization efficiency, Hallo4D incorporates motion-aware keyframe sampling, LMM-guided initialization, and appearance alignment. We additionally introduce exposure-aware optimization and visibility pruning to enhance robustness under challenging viewpoints. Extensive experiments demonstrate that Hallo4D consistently outperforms strong baselines across diverse 3D and 4D generation settings, providing a scalable and generalizable solution for consistency-aware content generation.
