Detecting Multimedia Generated by Large AI Models: A Survey
Department of Computer and Information Technology, Purdue University(普渡大学计算机与信息科技系);School of Software, Nanchang University(南昌大学软件学院);Amazon Prime Video(亚马逊Prime视频);Department of Epidemiology and Biostatistics, School of Public Health(公共卫生学院流行病学与生物统计学系);Department of Computer Science, College of Nanotechnology, Science, and Engineering(纳米技术、科学与工程学院计算机科学系);University at Albany, SUNY(阿尔巴尼大学)
Li Lin, Neeraj Gupta, Yue Zhang, Hainan Ren, Chun-Hao Liu, Feng Ding, Xin Wang, Xin Li, Luisa Verdoliva, Shu Hu
The rapid advancement of Large AI Models (LAIMs), particularly diffusion models and large language models, has marked a new era where AI-generated multimedia is increasingly integrated into various aspects of daily life. Although beneficial in numerous fields, this content presents significant risks, including potential misuse, societal disruptions, and ethical concerns. Consequently, detecting multimedia generated by LAIMs has become crucial, with a marked rise in related research. Despite this, there remains a notable gap in systematic surveys that focus specifically on detecting LAIM-generated multimedia. Addressing this, we provide the first survey to comprehensively cover existing research on detecting multimedia (such as text, images, videos, audio, and multimodal content) created by LAIMs. Specifically, we introduce a novel taxonomy for detection methods, categorized by media modality, and aligned with two perspectives: pure detection (aiming to enhance detection performance) and beyond detection (adding attributes like generalizability, robustness, and interpretability to detectors). Additionally, we have presented a brief overview of generation mechanisms, public datasets, online detection tools, and evaluation metrics to provide a valuable resource for researchers and practitioners in this field. Most importantly, we offer a focused analysis from a social media perspective to highlight their broader societal impact. Furthermore, we identify current challenges in detection and propose directions for future research that address unexplored, ongoing, and emerging issues in detecting multimedia generated by LAIMs. Our aim for this survey is to fill an academic gap and contribute to global AI security efforts, helping to ensure the integrity of information in the digital realm. The project link is https://github.com/Purdue-M2/Detect-LAIM-generated-Multimedia-Survey.
Instruction-Guided Editing Controls for Images and Multimedia: A Survey in LLM era
Griffith University(格里菲斯大学);University of Bremen(不来梅大学);Ho Chi Minh City University of Technology(胡志明市技术大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);Hanoi University of Science and Technology(河内科技大学);The University of Queensland(昆士兰大学)
Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey
University of California, Berkeley(加州大学伯克利分校);University of Science and Technology of China(中国科学技术大学);University of Southern California(南加州大学);IBM Research AI(IBM人工智能研究院);Rensselaer Polytechnic Institute(伦斯勒理工学院)
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Guangdong University of Technology(广东工业大学);University of Macau(澳门大学);Hefei University of Technology(合肥工业大学)
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Peng Cheng Laboratory(鹏城实验室)