Urbex:面向城市级3D场景的智能体空间定位
Urbex: Agentic Spatial Grounding for City-Scale 3D Scenes
AI总结:
本研究提出智能体视觉-语言框架Urbex,将城市级3D定位转化为主动空间搜索,在CityRefer数据集上取得最优指标且效率更高,跨场景鲁棒性也得到提升。
摘要:
城市级3D定位通常被表述为候选排序问题,需要预先计算实例提案,在大型室外场景中效率低下。我们提出Urbex,这是一种智能体视觉-语言框架,将定位转化为主动空间搜索。Urbex将城市级3D场景表示为交互式多视图环境,允许VLM智能体查询地标、放大局部区域、渲染斜视图并输出定位边界框。我们在CityRefer上使用强化学习优化该工具使用策略,采用平滑定位奖励和轻量级塑形奖励以实现高效探索。实验表明,Urbex在CityRefer上取得最佳Hit@1和IoU,支持无候选定位(无需真实边界框),且比最强的城市级排序基线快得多;在STPLS3D-Refer上的零样本结果进一步表明其跨场景鲁棒性得到提升。
英文摘要:
City-scale 3D grounding is commonly formulated as candidate ranking, which requires precomputed instance proposals and becomes inefficient in large outdoor scenes. We propose Urbex, an agentic vision-language framework that instead casts grounding as active spatial search. Urbex represents a city-scale 3D scene as an interactive multi-view environment, allowing a VLM agent to query landmarks, zoom into local regions, render oblique views, and commit a grounded bounding box. We optimize this tool-use policy with reinforcement learning on CityRefer, using smooth localization rewards and lightweight shaping rewards for efficient exploration. Experiments show that Urbex achieves the best Hit@1 and IoU on CityRefer, supports candidate-free grounding without ground-truth boxes, and is substantially faster than the strongest city-scale ranking baseline. Zero-shot results on STPLS3D-Refer further indicate improved cross-scene robustness.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。