面向问题的分割与检索:辅导对话的案例研究
Problem-Oriented Segmentation and Retrieval: Case Study on Tutoring Conversations
AI总结:
本文提出面向问题的分割与检索(POSR)任务,构建了首个真实辅导课程数据集LessonLink,并证明联合建模方法在分割和检索指标上显著优于独立流水线。
摘要:
许多开放式对话(例如辅导课程或商务会议)围绕预定义的参考资料展开,如工作表或会议要点。为了提供一个研究此类对话结构的框架,我们引入了面向问题的分割与检索(POSR),该任务将对话联合分解为多个片段,并将每个片段链接到相关的参考条目。作为案例研究,我们将POSR应用于教育领域,其中围绕问题有效构建课程结构至关重要但十分困难。我们提出了LessonLink,这是首个真实辅导课程数据集,包含3,500个片段,涵盖24,300分钟的教学时间,并链接到116道SAT数学问题。我们定义并评估了几种用于POSR的联合和独立方法,包括分割(例如TextTiling)、检索(例如ColBERT)以及大型语言模型(LLMs)方法。我们的结果强调,将POSR建模为一项联合任务至关重要:POSR方法在联合指标上比独立的分割和检索流水线高出多达76%,在分割指标上比传统分割方法高出多达78%。我们展示了POSR对下游教育应用的实际影响,为真实课程结构中的语言和时间使用提供了新的见解。
英文摘要:
Many open-ended conversations (e.g., tutoring lessons or business meetings) revolve around pre-defined reference materials, like worksheets or meeting bullets. To provide a framework for studying such conversation structure, we introduce Problem-Oriented Segmentation & Retrieval (POSR), the task of jointly breaking down conversations into segments and linking each segment to the relevant reference item. As a case study, we apply POSR to education where effectively structuring lessons around problems is critical yet difficult. We present LessonLink, the first dataset of real-world tutoring lessons, featuring 3,500 segments, spanning 24,300 minutes of instruction and linked to 116 SAT math problems. We define and evaluate several joint and independent approaches for POSR, including segmentation (e.g., TextTiling), retrieval (e.g., ColBERT), and large language models (LLMs) methods. Our results highlight that modeling POSR as one joint task is essential: POSR methods outperform independent segmentation and retrieval pipelines by up to +76% on joint metrics and surpass traditional segmentation methods by up to +78% on segmentation metrics. We demonstrate POSR's practical impact on downstream education applications, deriving new insights on the language and time use in real-world lesson structures.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。