AutoTour：基于智能手机和LLMs的自动照片导览系统
AutoTour: Automatic Photo Tour Guide with Smartphones and LLMs
AI总结：
AutoTour利用智能手机和LLMs自动为用户照片生成细粒度地标注释和描述，实现可扩展且上下文感知的交互式探索体验。
摘要：
我们提出了AutoTour，一个通过自动为用户拍摄的照片生成细粒度地标注释和描述性叙述来增强用户探索的系统。AutoTour的核心思想是将从照片中提取的视觉特征与从开放匹配数据库中查询的附近地理空间特征融合。与依赖预定义内容或专有数据集的现有旅游应用不同，AutoTour利用开放且可扩展的数据源，提供可扩展且上下文感知的照片引导。为此，我们设计了一个无需训练的流程，首先提取并过滤用户GPS位置周围的 relevant geospatial 特征。然后通过基于VLM的特征检测识别用户照片中的主要地标，并将其投影到水平空间平面上。几何匹配算法根据估计的距离和方向将照片特征与对应的地理空间实体对齐。匹配的特征随后被直接标注在原始照片上，并伴有大语言模型生成的文本和音频描述，以提供信息丰富、类似旅游的体验。我们证明了AutoTour能够为标志性和不为人知的地标提供丰富的、可解释的注释，从而实现一种新的交互式、上下文感知的探索形式，弥合了视觉感知和地理空间理解之间的差距。
英文摘要：
We present AutoTour, a system that enhances user exploration by automatically generating fine-grained landmark annotations and descriptive narratives for photos captured by users. The key idea of AutoTour is to fuse visual features extracted from photos with nearby geospatial features queried from open matching databases. Unlike existing tour applications that rely on pre-defined content or proprietary datasets, AutoTour leverages open and extensible data sources to provide scalable and context-aware photo-based guidance. To achieve this, we design a training-free pipeline that first extracts and filters relevant geospatial features around the user's GPS location. It then detects major landmarks in user photos through VLM-based feature detection and projects them into the horizontal spatial plane. A geometric matching algorithm aligns photo features with corresponding geospatial entities based on their estimated distance and direction. The matched features are subsequently grounded and annotated directly on the original photo, accompanied by large language model-generated textual and audio descriptions to provide an informative, tour-like experience. We demonstrate that AutoTour can deliver rich, interpretable annotations for both iconic and lesser-known landmarks, enabling a new form of interactive, context-aware exploration that bridges visual perception and geospatial understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
