基于互补深度与语义提示的资源高效可供性定位
Resource-Efficient Affordance Grounding with Complementary Depth and Semantic Prompts
AI总结：
本文提出 BiT-Align 框架，通过旁路提示模块将深度图像直接嵌入 RGB 编码器，并利用文本特征引导注意力选择，在显著降低参数量的同时提升可供性定位精度。
摘要：
可供性（Affordance）指智能体从环境中感知并利用的功能属性，是机器人执行动作所需的关键感知信息。这类信息本质上丰富且多模态。现有多模态可供性方法在提取有用信息方面存在局限，主要源于简单的结构设计、基础的融合方式以及庞大的模型参数，难以满足实际部署的性能要求。针对这些问题，本文提出了 BiT-Align 图像-深度-文本可供性映射框架。该框架包含旁路提示模块（Bypass Prompt Module，BPM）和文本特征引导（Text Feature Guidance，TFG）注意力选择机制。BPM 将辅助模态深度图像直接作为提示集成到主模态 RGB 图像中，并嵌入主模态编码器，无需引入额外编码器。这降低了模型参数量，并有效提升了功能区域定位精度。TFG 机制利用文本特征引导图像编码器中注意力头的选择与增强，改善了对可供性特征的理解。实验结果表明，所提方法在公开的 AGD20K 和 HICO-IIF 数据集上取得了显著的性能提升。在 AGD20K 数据集上，与当前最优方法相比，我们在 KLD 指标上实现了 6.0% 的提升，同时将模型参数量减少了 88.8%，展现出实际应用价值。源代码将在 https://github.com/DAWDSE/BiT-Align 公开发布。
英文摘要：
Affordance refers to the functional properties that an agent perceives and utilizes from its environment, and is key perceptual information required for robots to perform actions. This information is rich and multimodal in nature. Existing multimodal affordance methods face limitations in extracting useful information, mainly due to simple structural designs, basic fusion methods, and large model parameters, making it difficult to meet the performance requirements for practical deployment. To address these issues, this paper proposes the BiT-Align image-depth-text affordance mapping framework. The framework includes a Bypass Prompt Module (BPM) and a Text Feature Guidance (TFG) attention selection mechanism. BPM integrates the auxiliary modality depth image directly as a prompt to the primary modality RGB image, embedding it into the primary modality encoder without introducing additional encoders. This reduces the model's parameter count and effectively improves functional region localization accuracy. The TFG mechanism guides the selection and enhancement of attention heads in the image encoder using textual features, improving the understanding of affordance characteristics. Experimental results demonstrate that the proposed method achieves significant performance improvements on public AGD20K and HICO-IIF datasets. On the AGD20K dataset, compared with the current state-of-the-art method, we achieve a 6.0% improvement in the KLD metric, while reducing model parameters by 88.8%, demonstrating practical application values. The source code will be made publicly available at https://github.com/DAWDSE/BiT-Align.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
