360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user's future FoV not only based on the user's own past FoV center trajectory but also other users' future FoV locations. We propose multiple prediction models based on two different FoV representations: one using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users' FoVs are very helpful for improving long-term predictions.
StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA
Henan Institute of Advanced Technology, Zhengzhou University（河南高级技术研究所，郑州大学）；Institute of Automation, CAS（自动化研究所，中国科学院）；UCAS（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）
Streaming Video Generation with Streaming Force Control
Northeastern University（东北大学）；Impossible Research；University of California, Berkeley（加州大学伯克利分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts
Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；State Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室）；Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；X-LANCE Lab, Shanghai Jiao Tong University（上海交通大学X-LANCE实验室）
Learning Streaming Video Representation via Multitask Training
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Shanghai Innovation Institute（上海创新研究院）；Fudan University（复旦大学）；Shanghai AI Laboratory（上海人工智能实验室）
SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Kuaishou Technology（快手科技）；Zhengzhou University（郑州大学）；ShanghaiTech University（上海科技大学）；Peng Cheng Laboratory（鹏城实验室）
StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding
Tsinghua University（清华大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；OPPO AI Center, OPPO Inc.（OPPO人工智能中心，OPPO公司）
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；JD.COM（京东）
EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant
EPIC Lab, Shanghai Jiao Tong University（上海交通大学EPIC实验室）；Tsinghua University（清华大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Fudan University（复旦大学）
Online Streaming Video Super-Resolution with Convolutional Look-Up Table
Zhejiang University（浙江大学）；Tongji University（同济大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiao Tong University（上海交通大学）；Microsoft Research Asia（微软亚洲研究院）
Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding
Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University（清华大学交叉信息研究院（IIIS））；Nanjing University（南京大学）；University of Science and Technology of China（中国科学技术大学）；Microsoft Research Asia（微软亚洲研究院）
