基于LiveFood基准的领域增量式视频精彩片段检测探索
Exploring Domain Incremental Video Highlights Detection with the LiveFood Benchmark
AI总结：
针对现有视频精彩片段检测方法基于封闭世界假设、扩展性差的问题，提出基于参数化原型的增量学习方法GPE，构建含5100余条视频的LiveFood基准，在增量与经典数据集上均取得优异表现，首次探索该领域增量学习方向。
摘要：
视频精彩片段检测（VHD）是计算机视觉领域的活跃研究方向，旨在从原始视频输入中定位最吸引用户的片段。但多数VHD方法基于封闭世界假设，即预先定义固定数量的精彩片段类别，且所有训练数据均可提前获取，导致现有方法在面对不断增加的精彩片段领域和训练数据时扩展性较差。针对上述问题，我们提出一种名为Global Prototype Encoding（GPE，全局原型编码）的新型视频精彩片段检测方法，通过参数化原型进行增量学习以适配新领域。为推动这一新研究方向的发展，我们收集了一个标注精细的数据集LiveFood，包含超过5100条美食直播视频，涵盖食材、烹饪、摆盘、食用四个领域。据我们所知，这是首个在增量学习设置下探索视频精彩片段检测的工作，为将VHD应用于精彩片段关注领域与训练数据随时间持续增长的实际场景开辟了新方向。我们通过大量实验验证了GPE的有效性。值得注意的是，在LiveFood数据集上，GPE的表现优于主流领域增量学习方法，在所有领域上的mAP均实现显著提升。在经典数据集上，GPE也取得了与现有先进方法相当的性能。代码可在以下链接获取：https://github.com/ForeverPs/IncrementalVHD_GPE。
英文摘要：
Video highlights detection (VHD) is an active research field in computer vision, aiming to locate the most user-appealing clips given raw video inputs. However, most VHD methods are based on the closed world assumption, i.e., a fixed number of highlight categories is defined in advance and all training data are available beforehand. Consequently, existing methods have poor scalability with respect to increasing highlight domains and training data. To address above issues, we propose a novel video highlights detection method named Global Prototype Encoding (GPE) to learn incrementally for adapting to new domains via parameterized prototypes. To facilitate this new research direction, we collect a finely annotated dataset termed LiveFood, including over 5,100 live gourmet videos that consist of four domains: ingredients, cooking, presentation, and eating. To the best of our knowledge, this is the first work to explore video highlights detection in the incremental learning setting, opening up new land to apply VHD for practical scenarios where both the concerned highlight domains and training data increase over time. We demonstrate the effectiveness of GPE through extensive experiments. Notably, GPE surpasses popular domain incremental learning methods on LiveFood, achieving significant mAP improvements on all domains. Concerning the classic datasets, GPE also yields comparable performance as previous arts. The code is available at: https://github.com/ForeverPs/IncrementalVHD_GPE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
