Dynamic scene graphs generated from video clips could help enhance the semantic visual understanding in a wide range of challenging tasks such as environmental perception, autonomous navigation, and task planning of self-driving vehicles and mobile robots. In the process of temporal and spatial modeling during dynamic scene graph generation, it is particularly intractable to learn time-variant relations in dynamic scene graphs among frames. In this paper, we propose a Time-variant Relation-aware TRansformer (TR$^2$), which aims to model the temporal change of relations in dynamic scene graphs. Explicitly, we leverage the difference of text embeddings of prompted sentences about relation labels as the supervision signal for relations. In this way, cross-modality feature guidance is realized for the learning of time-variant relations. Implicitly, we design a relation feature fusion module with a transformer and an additional message token that describes the difference between adjacent frames. Extensive experiments on the Action Genome dataset prove that our TR$^2$ can effectively model the time-variant relations. TR$^2$ significantly outperforms previous state-of-the-art methods under two different settings by 2.1% and 2.6% respectively.
Generating Multimodal Driving Scenes via Next-Scene Prediction
School of Software Engineering, XJTU（西安交通大学软件学院）；Horizon Robotics（地平线机器人）；School of Computer and Communication Sciences, EPFL（洛桑联邦理工学院计算机与通信科学学院）；University of Chinese Academy of Sciences（中国科学院大学）
Self-Supervised Bird's Eye View Motion Prediction with Cross-Modality Signals
Shanghai Jiao Tong University（上海交通大学）；University of Chinese Academy of Sciences（中国科学院大学）；University of Southern California（南加州大学）；Shanghai AI Laboratory（上海人工智能实验室）
Fine-Grained Scene Image Classification with Modality-Agnostic Adapter
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Innovation Center of Calligraphy and Painting Creation Technology, MCT（中国传媒书画创作技术创新中心）；Videt Technology（维德科技）
All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmark
Guangdong-HongKong-Macao Joint Laboratory for Intelligent MicroNano Optoelectronic Technology（粤港澳联合智能微纳光电子技术实验室）；School of Physics and Optoelectronic Engineering, Foshan University（佛山大学物理与光电工程学院）；Guangdong Provincial Key Laboratory of Industrial Intelligent Inspection Technology（广东省工业智能检测技术重点实验室）；School of Instrumentation and Optoelectronic Engineering, Beihang University（北京航空航天大学仪器与光电工程学院）；School of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）；School of Artificial Intelligence, Optics and Electronics (i0PEN), School of Computer Science, Northwestern Polytechnical University（人工智能、光学与电子学院（i0PEN），西北工业大学计算机学院）
Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation
Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院人形与机器人研究所）；Mercedes-Benz Tech Innovation（梅赛德斯-奔驰技术创新公司）；School of Robotics, Hunan University（湖南大学机器人学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学机器人视觉感知与控制技术国家工程研究中心）
A Constrained Deformable Convolutional Network for Efficient Single Image Dynamic Scene Blind Deblurring with Spatially-Variant Motion Blur Kernels Estimation
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Chongqing University（重庆大学）
AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception
Guangdong-HongKong-Macao Joint Laboratory for Intelligent MicroNano Optoelectronic Technology（粤港澳联合智能微纳光电子技术联合实验室）；Foshan University（佛山大学）；School of Mechanical Electronic and Information Engineering（机械电子信息工程学院）；China University of Mining and Technology（中国矿业大学）；School of Information Engineering and Automation（信息工程与自动化学院）
Reducing Domain Gap in Frequency and Spatial domain for Cross-modality Domain Adaptation on Medical Image Segmentation
Digital Medical Research Center, School of Basic Medical Science, Fudan University（复旦大学基础医学院数字医学研究中心）；Shanghai Key Lab of Medical Image Computing and Computer Assisted Intervention（上海市医学影像计算与计算机辅助干预重点实验室）
ViTGaze: Gaze Following with Interaction Features in Vision Transformers
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）
提出单模态 gaze following 框架 ViTGaze，利用 ViT self-attention 中的 inter-token 交互提取人-场景交互信息，在减少参数的同时实现了 SOTA 性能。
Cross-Modality Gait Recognition: Bridging LiDAR and Camera Modalities for Human Identification
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；University of Cordoba（科尔多瓦大学）；The Hong Kong Polytechnic University（香港理工大学）
WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localization
School of Computer Engineering and Science, Shanghai University, China（上海大学计算机工程与科学学院）；Faculty of Science and Technology and Institute of Collaborative Innovation, University of Macau, China（澳门大学科技学院和协同创新研究院）
AV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answering
Great Bay University（大湾区大学）；Dongguan Key Laboratory for Intelligence and Information Technology（东莞智能与信息科技重点实验室）；Nankai University（南开大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；The Chinese University of Hong Kong（香港中文大学）
