Exploring Real-Time Super-Resolution: Benchmarking and Fine-Tuning for Streaming Content
Lomonosov Moscow State University(罗蒙诺索夫莫斯科国立大学);AI Center, Lomonosov Moscow State University(Lomonosov莫斯科国立大学人工智能中心);MSU Institute for Artificial Intelligence, Lomonosov Moscow State University(MSU人工智能研究所,Lomonosov莫斯科国立大学)
Evgeney Bogatyrev, Khaled Abud, Ivan Molodetskikh, Nikita Alutis, Dmitriy Vatolin
Recent advancements in real-time super-resolution have enabled higher-quality video streaming, yet existing methods struggle with the unique challenges of compressed video content. Commonly used datasets do not accurately reflect the characteristics of streaming media, limiting the relevance of current benchmarks. To address this gap, we introduce a comprehensive dataset - StreamSR - sourced from YouTube, covering a wide range of video genres and resolutions representative of real-world streaming scenarios. We benchmark 11 state-of-the-art real-time super-resolution models to evaluate their performance for the streaming use-case.
Furthermore, we propose EfRLFN, an efficient real-time model that integrates Efficient Channel Attention and a hyperbolic tangent activation function - a novel design choice in the context of real-time super-resolution. We extensively optimized the architecture to maximize efficiency and designed a composite loss function that improves training convergence. EfRLFN combines the strengths of existing architectures while improving both visual quality and runtime performance.
Finally, we show that fine-tuning other models on our dataset results in significant performance gains that generalize well across various standard benchmarks. We made the dataset, the code, and the benchmark available at https://github.com/EvgeneyBogatyrev/EfRLFN.
APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants
Shanghai Jiao Tong University(上海交通大学);Eastern Institute of Technology, Ningbo(宁波东方理工大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Zhongguancun Academy(中关村学院);Dalian University of Technology(大连理工大学);Beihang University(北京航空航天大学);Hong Kong Polytechnic University(香港理工大学)
Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing
Beihang University, Beijing, China(北京航空航天大学);University of Liverpool, Liverpool, UK(利物浦大学);Institute of Acoustics, Chinese Academy of Sciences, Beijing, China(中国科学院声学研究所)
Viewport Prediction for Volumetric Video Streaming by Exploring Video Saliency and Trajectory Information
School of Computer Science and Information Engineering, Hefei University of Technology(计算机科学与信息工程学院,合肥工业大学);The University of Electro-Communications(电子通信大学);Aarhus University(奥胡斯大学);School of Electrical Engineering and Automation, Hefei University of Technology(电气工程与自动化学院,合肥工业大学);Engineering Technology Research Center of Industrial Automation of Anhui Province(安徽省工业自动化工程技术研究中心);School of Information and Electronics, Beijing Institute of Technology(信息与电子学院,北京理工大学)
LiveStar: Live Streaming Assistant for Real-World Online Video Understanding
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);ShanghaiTech University(上海科技大学);Kuaishou Technology(快手科技);Peng Cheng Laboratory(鹏城实验室)