FineBio：一个带有层级标注的生物实验细粒度视频数据集
FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation
AI总结：
本研究提出带有层级标注的细粒度生物实验视频数据集FineBio，包含32名参与者14.5小时多视角模拟实验视频，标注了实验方案、步骤、原子操作等多层级信息，并给出四项相关任务的基线结果，为结构化行为理解与手-物交互研究提供新基准。
摘要：
在科学发展进程中，对实验过程进行准确且可复现的记录至关重要。从视频中自动识别实验操作动作，能够补充实验记录，为实验人员提供帮助。为实现这一目标，我们提出了FineBio——一个全新的、记录人员开展生物实验过程的细粒度视频数据集。该数据集包含32名参与者进行模拟生物实验的多视角视频，总时长为14.5小时。单个实验呈层级结构：一个实验方案（protocol）包含若干步骤（step），每个步骤又可进一步拆解为一组原子操作（atomic operation）。生物实验的独特性在于，尽管实验要求严格遵循每个实验方案中描述的步骤，但原子操作的执行顺序存在灵活性。我们针对实验方案、步骤、原子操作、物体位置及其操作状态提供了层级标注，为结构化行为理解与手-物交互识别带来了新的挑战。为明确生物实验场景下行为理解的难点，我们引入了基线模型，并给出了四项不同任务的实验结果，包括：(i) 步骤分割、(ii) 原子操作检测、(iii) 物体检测，以及(iv) 被操作/受影响物体检测。数据集与代码可通过https://github.com/aistairc/FineBio获取。
英文摘要：
In the development of science, accurate and reproducible documentation of the experimental process is crucial. Automatic recognition of the actions in experiments from videos would help experimenters by complementing the recording of experiments. Towards this goal, we propose FineBio, a new fine-grained video dataset of people performing biological experiments. The dataset consists of multi-view videos of 32 participants performing mock biological experiments with a total duration of 14.5 hours. One experiment forms a hierarchical structure, where a protocol consists of several steps, each further decomposed into a set of atomic operations. The uniqueness of biological experiments is that while they require strict adherence to steps described in each protocol, there is freedom in the order of atomic operations. We provide hierarchical annotation on protocols, steps, atomic operations, object locations, and their manipulation states, providing new challenges for structured activity understanding and hand-object interaction recognition. To find out challenges on activity understanding in biological experiments, we introduce baseline models and results on four different tasks, including (i) step segmentation, (ii) atomic operation detection (iii) object detection, and (iv) manipulated/affected object detection. Dataset and code are available from https://github.com/aistairc/FineBio.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
