WinDB：无需HMD且无畸变的全景视频注视学习
WinDB: HMD-free and Distortion-free Panoptic Video Fixation Learning
AI总结：
针对HMD采集全景注视存在盲带的问题，提出无HMD、无畸变的WinDB采集方法，发布PanopticVideo-300数据集，并用FishNet建模频繁注视转移。
摘要：
迄今为止，在全景视频中执行注视点采集的广泛采用方式基于头戴式显示器（HMD）：用户佩戴HMD自由探索给定全景场景，同时采集其注视点。然而，当给定全景场景包含间歇性显著事件时，这种被广泛使用的数据采集方法不足以训练深度模型准确预测其中哪些区域最重要。主要原因在于，使用HMD采集注视点时始终存在“盲带”，因为用户无法一直转动头部来探索整个全景场景。因此，采集到的注视点往往被困在某些局部视图中，其余区域则成为“盲带”。所以，使用基于HMD的方法、通过累积局部视图采集的注视数据，无法准确表示复杂全景场景的整体全局重要性——而这正是注视点的主要目的。为克服该问题，本文提出用于全景视频的带动态模糊的辅助窗口（WinDB）注视采集方法，该方法不需要HMD，并能够很好地反映区域级重要程度。利用我们的WinDB方法，我们发布了新的PanopticVideo-300数据集，包含300个全景片段，覆盖超过225个类别。具体而言，由于使用WinDB采集注视点不存在盲带，我们的新数据集中存在频繁且强烈的“注视转移”——这是一种长期被以往研究忽视的非常特殊的现象。因此，我们提出有效的注视转移网络（FishNet）来解决这一问题。所有这些新的注视采集工具、数据集和网络都极具潜力，有望为360°环境中与注视相关的研究和应用开启新时代。
英文摘要：
To date, the widely adopted way to perform fixation collection in panoptic video is based on a head-mounted display (HMD), where users' fixations are collected while wearing an HMD to explore the given panoptic scene freely. However, this widely-used data collection method is insufficient for training deep models to accurately predict which regions in a given panoptic are most important when it contains intermittent salient events. The main reason is that there always exist "blind zooms" when using HMD to collect fixations since the users cannot keep spinning their heads to explore the entire panoptic scene all the time. Consequently, the collected fixations tend to be trapped in some local views, leaving the remaining areas to be the "blind zooms". Therefore, fixation data collected using HMD-based methods that accumulate local views cannot accurately represent the overall global importance - the main purpose of fixations - of complex panoptic scenes. To conquer, this paper introduces the auxiliary window with a dynamic blurring (WinDB) fixation collection approach for panoptic video, which doesn't need HMD and is able to well reflect the regional-wise importance degree. Using our WinDB approach, we have released a new PanopticVideo-300 dataset, containing 300 panoptic clips covering over 225 categories. Specifically, since using WinDB to collect fixations is blind zoom free, there exists frequent and intensive "fixation shifting" - a very special phenomenon that has long been overlooked by the previous research - in our new set. Thus, we present an effective fixation shifting network (FishNet) to conquer it. All these new fixation collection tool, dataset, and network could be very potential to open a new age for fixation-related research and applications in 360o environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
