一个面向时空敏感的兴趣点问答数据集
A Dataset for Spatiotemporal-Sensitive POI Question Answering
AI总结：
针对现有问答数据集缺乏时空敏感问题、难以评估模型时空推理能力的问题，研究构建了以兴趣点为核心的POI-QA双语时空敏感问答数据集，实验表明当前顶尖多语种大语言模型在该数据集上表现远逊于人类，可为相关算法发展提供可靠基准。
摘要：
时空关系在数据科学中至关重要，因为许多预测和推理任务需要跨空间和时间维度进行分析——例如，在陌生城市中导航需要规划行程，对地点进行排序并安排文化体验的时间。然而，现有的问答（QA）数据集缺乏足够的时空敏感问题，因此无法作为评估模型时空推理能力的合适基准。为填补这一空白，我们提出了POI-QA，这是一个以兴趣点（Point of Interest, POI）为核心的新型时空敏感问答数据集，通过三个关键步骤构建：挖掘GAIA的开源车辆轨迹数据并将其与高精度地理POI数据对齐，对含噪声的时空事实进行严格的人工验证，生成反映人类可理解的时空推理任务的双语（中文/英文）问答对。我们的数据集对模型解析复杂时空依赖关系的能力提出了挑战，对当前最先进的多语种大语言模型（如Qwen2.5-7B、Llama3.1-8B）的评估显示出明显的局限性：即使是表现最佳的模型（经RAG+LoRA微调的Qwen2.5-7B），在最简单的任务上的前10命中率（HR@10）也仅为0.41，远低于人类0.56的表现。这凸显了大语言模型在执行一致的时空推理方面仍存在持续的弱点，同时也表明POI-QA是推动时空动态敏感算法发展的可靠基准。该数据集已在https://www.kaggle.com/ds/7394666公开发布。
英文摘要：
Spatiotemporal relationships are critical in data science, as many prediction and reasoning tasks require analysis across both spatial and temporal dimensions--for instance, navigating an unfamiliar city involves planning itineraries that sequence locations and timing cultural experiences. However, existing Question-Answering (QA) datasets lack sufficient spatiotemporal-sensitive questions, making them inadequate benchmarks for evaluating models' spatiotemporal reasoning capabilities. To address this gap, we introduce POI-QA, a novel spatiotemporal-sensitive QA dataset centered on Point of Interest (POI), constructed through three key steps: mining and aligning open-source vehicle trajectory data from GAIA with high-precision geographic POI data, rigorous manual validation of noisy spatiotemporal facts, and generating bilingual (Chinese/English) QA pairs that reflect human-understandable spatiotemporal reasoning tasks. Our dataset challenges models to parse complex spatiotemporal dependencies, and evaluations of state-of-the-art multilingual LLMs (e.g., Qwen2.5-7B, Llama3.1-8B) reveal stark limitations: even the top-performing model (Qwen2.5-7B fine-tuned with RAG+LoRA) achieves a top 10 Hit Ratio (HR@10) of only 0.41 on the easiest task, far below human performance at 0.56. This underscores persistent weaknesses in LLMs' ability to perform consistent spatiotemporal reasoning, while highlighting POI-QA as a robust benchmark to advance algorithms sensitive to spatiotemporal dynamics. The dataset is publicly available at https://www.kaggle.com/ds/7394666.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
