Automatically understanding video contents is important for several applications in Civic Monitoring (CM), general Surveillance (SL), Assisted Living (AL), etc. Decades of Image and Video Analysis (IVA) research have advanced tasks such as content extraction (e.g., object recognition and tracking). Identifying meaningful activities or situations (e.g., two objects coming closer) remains difficult and cannot be achieved by content extraction alone. Currently, Video Situation Analysis (VSA) is done manually with a human in the loop, which is error-prone and labor-intensive, or through custom algorithms designed for specific video types or situations. These algorithms are not general-purpose and require a new algorithm/software for each new situation or video from a new domain.
This report proposes a general-purpose VSA framework that overcomes the above limitations. Video contents are extracted once using state-of-the-art Video Content Extraction technologies. They are represented using two alternative models -- the extended relational model (R++) and graph models. When represented using R++, the extracted contents can be used as data streams, enabling Continuous Query Processing via the proposed Continuous Query Language for Video Analysis. The graph models complement this by enabling the detection of situations that are difficult or impossible to detect using the relational model alone. Existing graph algorithms and newly developed algorithms support a wide variety of situation detection. To support domain independence, primitive situation variants across domains are identified and expressed as parameterized templates. Extensive experiments were conducted across several interesting situations from three domains -- AL, CM, and SL-- to evaluate the accuracy, efficiency, and robustness of the proposed approach using a dataset of videos of varying lengths from these domains.
FDIM: A Feature-distance-based Generic Video Quality Metric for Versatile Codecs
College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息科学与电子工程学院);Zhejiang Key Laboratory of Multimodal Communication Networks and Intelligent Information Processing(浙江省多模态通信网络与智能信息处理重点实验室);Central Media Technology Institute, Huawei(华为中央媒体技术研究所)
Neural video codecs quality assessment dataset and benchmark
AI Center, Lomonosov Moscow State University(罗蒙诺索夫莫斯科国立大学人工智能中心);MSU Institute for Artificial Intelligence(莫斯科国立大学人工智能研究所);Lomonosov Moscow State University(罗蒙诺索夫莫斯科国立大学)
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Ningbo Institute of Digital Twin, Eastern Institute of Technology(东方理工大学宁波数字孪生研究院);Shenzhen Tsinghua Research Institute(深圳清华大学研究院)
From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所);SGIT AI Lab, State Grid Corporation of China(国家电网公司SGIT人工智能实验室);Zhejiang University of Technology(浙江工业大学);Baidu(百度)
Towards an Automated Multimodal Approach for Video Summarization: Building a Bridge Between Text, Audio and Facial Cue-Based Summarization
Center for Machine Vision and Signal Analysis, Faculty of ITEE, University of Oulu, Finland(机器视觉与信号分析中心,ITEE学院,奥卢大学,芬兰);Center for Machine Vision(机器视觉与信号分析中心);Signal Analysis, Faculty of ITEE, University of Oulu, Finland(信号分析,ITEE学院,奥卢大学,芬兰)