Visual signals in a video can be divided into content and motion. While content specifies which objects are in the video, motion describes their dynamics. Based on this prior, we propose the Motion and Content decomposed Generative Adversarial Network (MoCoGAN) framework for video generation. The proposed framework generates a video by mapping a sequence of random vectors to a sequence of video frames. Each random vector consists of a content part and a motion part. While the content part is kept fixed, the motion part is realized as a stochastic process. To learn motion and content decomposition in an unsupervised manner, we introduce a novel adversarial learning scheme utilizing both image and video discriminators. Extensive experimental results on several challenging datasets with qualitative and quantitative comparison to the state-of-the-art approaches, verify effectiveness of the proposed framework. In addition, we show that MoCoGAN allows one to generate videos with same content but different motion as well as videos with different content and same motion.
COMUNI: Decomposing Common and Unique Video Signals for Diffusion-based Video Generation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
本文提出 COMUNI 框架，通过 CU-VAE 自监督解耦视频的共性与独特信号，并用双流 CU-LDM 及联合位置嵌入实现高效、一致且连贯的扩散视频生成。
Decomposing Normal and Abnormal Features of Medical Images into Discrete Latent Codes for Content-Based Image Retrieval
National Cancer Center Research Institute（国立癌症中心研究所）；RIKEN Center for Advanced Intelligent Project（理化学研究所先进智能项目中心）；The University of Tokyo（东京大学）；National Cancer Center Hospital（国立癌症中心医院）
Decomposing Normal and Abnormal Features of Medical Images for Content-based Image Retrieval
National Cancer Center Research Institute（国立癌症研究中心研究所）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息理工学研究生院）；Research Center for Advanced Science and Technology, The University of Tokyo（东京大学先端科学技术研究中心）
Nuclear Diffusion Models for Low-Rank Background Suppression in Videos
Dept. of Electrical Engineering, Eindhoven University of Technology, the Netherlands（埃因霍温理工大学电气工程系，荷兰）；Philips Research North America, Cambridge MA, USA（飞利浦北美研究部，马萨诸塞州剑桥）
Next Block Prediction: Video Generation via Semi-Autoregressive Modeling
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室）；Microsoft Research（微软研究院）
Partitioning Image Representation in Contrastive Learning
Handong Global University（韩东全球大学）
在图像域中的对比学习中，锚样本和正样本被强制具有尽可能接近的表示。然而，强制两个样本具有相同的表示可能会产生误导，因为数据增强技术使两个样本不同。在本文中，我们介绍了一种新的表示，分区表示，它可以学习共同的和独特的特征的锚和积极的样本在对比学习。分区表示由两部分组成：内容部分和样式部分。内容部分表示类的共同特征，样式部分表示每个样本自身的特征，这可以导致数据增强方法的表示。我们可以简单地通过将对比学习的损失函数分解为分别关于两个单独表示的两个项来实现分区表示。为了评估我们的表示与两个部分，我们采取了两个框架模型：变分自编码器（VAE）和BootstrapYour Own Latent（BYOL），以显示内容和风格的可分性，并确认分类的泛化能力，分别。实验结果表明，该方法能够在VAE框架下分离两类信息，在线性可分性和下游少样本学习任务方面优于传统的BYOL方法.
