从文本分割到智能章节划分:一个用于结构化视频转录文本的新基准
From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions
AI总结:
本文提出面向非结构化口语视频转录文本的分割基准YTSeg和高效分层分割模型MiniSeg,并将文本分割扩展为包含片段标题生成与实时应用潜力的智能章节划分任务。
摘要:
文本分割是自然语言处理中的一项基础任务,其目标是将文档切分为连续的片段。然而,该领域以往的研究受到数据集有限的制约,这些数据集要么规模较小,要么是人工合成的,要么只包含结构良好的文档。在本文中,我们通过引入一个名为YTSeg的新基准来解决这些局限性,该基准聚焦于口语内容,这类内容本质上更加非结构化,并且在主题和结构上都更加多样。作为这项工作的一部分,我们提出了一个高效的分层分割模型MiniSeg,其性能优于当前最先进的基线模型。最后,我们将文本分割的概念扩展到一个更实用的“智能章节划分”任务,该任务涉及非结构化内容的分割、有意义片段标题的生成,以及模型在实时场景中的潜在应用。
英文摘要:
Text segmentation is a fundamental task in natural language processing, where documents are split into contiguous sections. However, prior research in this area has been constrained by limited datasets, which are either small in scale, synthesized, or only contain well-structured documents. In this paper, we address these limitations by introducing a novel benchmark YTSeg focusing on spoken content that is inherently more unstructured and both topically and structurally diverse. As part of this work, we introduce an efficient hierarchical segmentation model MiniSeg, that outperforms state-of-the-art baselines. Lastly, we expand the notion of text segmentation to a more practical "smart chaptering" task that involves the segmentation of unstructured content, the generation of meaningful segment titles, and a potential real-time application of the models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。