当F1失效时：面向对话主题分段的粒度感知评估
When F1 Fails: Granularity-Aware Evaluation for Dialogue Topic Segmentation
AI总结：
本文提出粒度感知评估框架，用于对话主题分段，强调粒度选择而非单一边界预测。
摘要：
对话主题分段支持摘要、检索、内存管理和会话连续性。尽管数十年的研究，评估实践仍由严格的边界匹配和基于F1的指标主导。现代基于大语言模型（LLM）的对话系统越来越多地依赖分段来管理超出固定上下文窗口的对话历史。在这些系统中，无结构的上下文积累会降低效率和连贯性。本文介绍了一个评估框架，该框架报告边界密度和分段对齐诊断（纯度和覆盖度）以及窗口容忍的F1（W-F1）。通过将边界评分与边界选择分开，我们评估了在不同密度范围内分段的质量，而不是在单一操作点上。跨数据集评估显示，报告的性能差异往往反映标注粒度不匹配，而不是单纯边界位置质量。我们评估了结构上不同的分段策略，涵盖八个对话数据集，涵盖任务导向、开放域、会议风格和合成交互。基于边界的指标与边界密度紧密相关：阈值扫描产生的W-F1变化比在不同方法之间切换更大。这些发现支持将主题分段视为粒度选择问题，而不是预测单一正确边界集。这促使将边界评分与边界选择分开，以分析和调整在不同标注粒度下的分段。
英文摘要：
Dialogue topic segmentation supports summarization, retrieval, memory management, and conversational continuity. Despite decades of work, evaluation practice remains dominated by strict boundary matching and F1-based metrics. Modern large language model (LLM) based conversational systems increasingly rely on segmentation to manage conversation history beyond fixed context windows. In such systems, unstructured context accumulation degrades efficiency and coherence. This paper introduces an evaluation framework that reports boundary density and segment alignment diagnostics (purity and coverage) alongside window-tolerant F1 (W-F1). By separating boundary scoring from boundary selection, we evaluate segmentation quality across density regimes rather than at a single operating point. Cross-dataset evaluation shows that reported performance differences often reflect annotation granularity mismatch rather than boundary placement quality alone. We evaluate structurally distinct segmentation strategies across eight dialogue datasets spanning task-oriented, open-domain, meeting-style, and synthetic interactions. Boundary-based metrics are strongly coupled to boundary density: threshold sweeps produce larger W-F1 changes than switching between methods. These findings support viewing topic segmentation as a granularity selection problem rather than prediction of a single correct boundary set. This motivates separating boundary scoring from boundary selection for analyzing and tuning segmentation under varying annotation granularities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
