无需训练的多模态大语言模型密集手部接触估计
Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models
AI总结：
本文提出ContactPrompt，利用多模态大语言模型进行无需训练的密集手部接触估计，通过引入详细的部位分割和部分顶点网格表示，结合多阶段结构化接触推理，实现高精度的接触预测。
摘要：
密集手部接触估计需要高水平的语义理解和精细的几何推理来准确定位接触区域。最近，多模态大语言模型（MLLMs）在理解视觉语义方面表现出强大能力，但将其用于密集手部接触估计仍处于探索阶段。本文提出ContactPrompt，一种无需训练的零样本方法，通过引入详细的部位分割和部分顶点网格表示，结合多阶段结构化接触推理，有效利用MLLMs的推理能力，实现了高精度的密集手部接触估计。令人惊讶的是，该方法在无需训练的情况下超越了之前在大规模密集接触数据集上训练的监督方法。
英文摘要：
Dense hand contact estimation requires both high-level semantic understanding and fine-grained geometric reasoning of human interaction to accurately localize contact regions. Recently, multi-modal large language models (MLLMs) have demonstrated strong capabilities in understanding visual semantics, enabled by vision-language priors learned from large-scale data. However, leveraging MLLMs for dense hand contact estimation remains underexplored. There are two major challenges in applying MLLMs to dense hand contact estimation. First, encoding explicit 3D hand geometry is difficult, as MLLMs primarily operate on vision and language modalities. Second, capturing fine-grained vertex-level contact remains challenging, as MLLMs tend to focus on high-level semantics rather than detailed geometric reasoning. To address these challenges, we propose ContactPrompt, a training-free and zero-shot approach for dense hand contact estimation using MLLMs. To effectively encode 3D hand geometry, we introduce a detailed hand-part segmentation and a part-wise vertex-grid representation that provides structured, localized geometric information. To enable accurate and efficient dense contact prediction, we develop a multi-stage structured contact reasoning with part conditioning, progressively bridging global semantics and fine-grained geometry. Therefore, our method effectively leverages the reasoning capabilities of MLLMs while enabling precise dense hand contact estimation. Surprisingly, the proposed approach outperforms previous supervised methods trained on large-scale dense contact datasets without requiring any training. The codes will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
