arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.15225cs.HC
收藏

UrbanClipAtlas:面向城市视频中事件和场景检索的视觉分析框架

UrbanClipAtlas: A Visual Analytics Framework for Event and Scene Retrieval in Urban Videos

Fundação Getulio Vargas(巴西瓦格蒂奥·富尔加斯基金会);New York University(纽约大学)

Joel Perca, Luis Sante, Juanpablo Heredia, Joao Rulff, Claudio Silva, Jorge Poco

AI总结:

本文提出UrbanClipAtlas框架,结合RAG、实体提取和视频定位技术,实现城市视频中事件和场景的高效检索与解释,通过知识图谱和增强聊天界面提升分析效率。

摘要:

从长时间的城市视频中提取可操作的见解通常是劳动密集型的:分析师必须手动筛选原始视频以定位目标事件或发现更广泛的行为趋势。本文提出了URBANCLIPATLAS,一种用于探索长城市视频的视觉分析系统,该系统结合检索增强生成(RAG)、领域感知实体提取和视频定位,以支持事件检索和解释。系统将长视频分割为短片段,使用视觉语言模型生成文本描述,并对这些片段进行语义检索。知识图谱将LLM答案中的实体和关系映射到领域特定的分类,并与检测到的对象和轨迹对齐,以支持视觉定位和验证。URBANCLIPATLAS通过增强的聊天界面支持场景检索,并通过紧密对齐文本输出与视频证据来改进场景解释。这种设计加强了文本推理与视觉证据之间的联系,减少了验证模型输出和改进假设所需的努力。我们通过StreetAware数据集上的两个案例研究展示了URBANCLIPATLAS的实用性,涉及危险场景和交叉动态。URBANCLIPATLAS帮助分析师在大规模城市视频集合中分析安全性和移动性相关模式。

英文摘要:

Extracting actionable insights from long-duration urban videos is often labor-intensive: analysts must manually sift through raw footage to pinpoint target events or uncover broader behavioral trends. In this work, we present URBANCLIPATLAS, a visual analytics system for exploring long urban videos recorded at street intersections. URBANCLIPATLAS combines retrieval-augmented generation (RAG), taxonomy-aware entity extraction, and video grounding to support event retrieval and interpretation. The system segments extended recordings into short clips, generates textual descriptions with a vision-language model, and indexes them for semantic retrieval. A knowledge graph maps entities and relations from LLM answers onto a domain-specific taxonomy and aligns them with detected objects and trajectories to support visual grounding and verification. URBANCLIPATLAS supports scene retrieval through an augmented chat-based interface and improves scene interpretation by tightly aligning textual outputs with video evidence. This design strengthens the connection between textual reasoning and visual evidence, reducing the effort required to validate model outputs and refine hypotheses. We demonstrate the usefulness of URBANCLIPATLAS on the StreetAware dataset through two case studies involving hazardous scenarios and crossing dynamics at street intersections. URBANCLIPATLAS helps analysts reason about safety- and mobility-related patterns across large urban video collections.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑