In this paper, the problem of head movement prediction for virtual reality videos is studied. In the considered model, a deep learning network is introduced to leverage position data as well as video frame content to predict future head movement. For optimizing data input into this neural network, data sample rate, reduced data, and long-period prediction length are also explored for this model. Simulation results show that the proposed approach yields 16.1\% improvement in terms of prediction accuracy compared to a baseline approach that relies only on the position data.
University of Science and Technology of China（中国科学技术大学）
In no-reference 360-degree image quality assessment (NR 360IQA), graph convolutional networks (GCNs), which model interactions between viewports through graphs, have achieved impressive performance. However, prevailing GCN-based NR 360IQA methods suffer from three main limitations. First, they only use high-level features of the distorted image to regress the quality score, while the human visual system (HVS) scores the image based on hierarchical features. Second, they simplify complex high-order interactions between viewports in a pairwise fashion through graphs. Third, in the graph construc…
Viewport Prediction for Volumetric Video Streaming by Exploring Video Saliency and Trajectory Information
School of Computer Science and Information Engineering, Hefei University of Technology（计算机科学与信息工程学院，合肥工业大学）；The University of Electro-Communications（电子通信大学）；Aarhus University（奥胡斯大学）；School of Electrical Engineering and Automation, Hefei University of Technology（电气工程与自动化学院，合肥工业大学）；Engineering Technology Research Center of Industrial Automation of Anhui Province（安徽省工业自动化工程技术研究中心）；School of Information and Electronics, Beijing Institute of Technology（信息与电子学院，北京理工大学）
Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessment
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；The University of Tokyo（东京大学）；University of Maryland, College Park（马里兰大学帕克分校）；Pingan Group（平安集团）
Geo-tagged photo based tourist attraction recommendation can discover users' travel preferences from their taken photos, so as to recommend suitable tourist attractions to them. However, existing visual content based methods cannot fully exploit the user and tourist attraction information of photos to extract visual features, and do not differentiate the significances of different photos. In this paper, we propose multi-level visual similarity based personalized tourist attraction recommendation using geo-tagged photos (MEAL). MEAL utilizes the visual contents of photos and interaction behavio…
A Survey on Personalized Content Synthesis with Diffusion Models
Department of Computing（计算系）；The Hong Kong Polytechnic University（香港理工大学）；Center for Artificial Intelligence and Robotics（人工智能与机器人中心）；Hong Kong Institute of Science & Innovation（香港科学创新研究院）；Chinese Academy of Sciences（中国科学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems（多模态人工智能系统国家重点实验室）；Chinese Academy of Sciences Institute of Automation（中国科学院自动化研究所）；School of Artificial Intelligence（人工智能学院）；University of Chinese Academy of Sciences（中国科学院大学）
Shanghai Jiao Tong University（上海交通大学）；Beijing University of Technology（北京工业大学）；University of Macau（澳门大学）
Augmented reality (AR) overlays digital content onto the reality. In AR system, correct and precise estimations of user's visual fixations and head movements can enhance the quality of experience by allocating more computation resources on the areas of interest. However, there is inadequate research about understanding the visual exploration of users when using an AR system or modeling AR visual attention. To bridge the gap between the saliency prediction on real-world scene and on scene augmented by virtual information, we construct the ARVR saliency dataset with 12 diverse videos viewed by 2…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The video broadcasting industry has been growing significantly in the recent years, specially on delivering personalized contents to the end users. While video broadcasting has continued to grow beyond TV, video adverting has become a key marketing tool to deliver targeted messages directly to the audience. However, unfortunately for broadband TV, a key problem is that the TV commercials target the broad audience, therefore lacking user-specific and personalized ad contents. In this paper, we propose a deep edge-cloud ad-placement system, and briefly describe our methodologies and the architec…
Fragile by Design: On the Limits of Adversarial Defenses in Personalized Generation
Department of Computer Science, University of Liverpool, Liverpool, L69 3BX, UK（利兹大学计算机科学系）；WMG, University of Warwick, Coventry, CV4 7AL, UK（沃里克大学WMG）
School of Computer Science, Fudan University（复旦大学计算机学院）；Shanghai Key Laboratory of Data Science, Fudan University（复旦大学上海市数据科学重点实验室）；Department of Computing, Imperial College London（帝国理工学院计算机系）；University of Colorado Boulder（科罗拉多大学博尔德分校）；University of Michigan（密歇根大学）
This work presents MemX: a biologically-inspired attention-aware eyewear system developed with the goal of pursuing the long-awaited vision of a personalized visual Memex. MemX captures human visual attention on the fly, analyzes the salient visual content, and records moments of personal interest in the form of compact video snippets. Accurate attentive scene detection and analysis on resource-constrained platforms is challenging because these tasks are computation and energy intensive. We propose a new temporal visual attention network that unifies human visual attention tracking and salient…
