Human intention prediction is a growing area of research where an activity in a video has to be anticipated by a vision-based system. To this end, the model creates a representation of the past, and subsequently, it produces future hypotheses about upcoming scenarios. In this work, we focus on pedestrians' early intention prediction in which, from a current observation of an urban scene, the model predicts the future activity of pedestrians that approach the street. Our method is based on a multi-modal transformer that encodes past observations and produces multiple predictions at different anticipation times. Moreover, we propose to learn the attention masks of our transformer-based model (Temporal Adaptive Mask Transformer) in order to weigh differently present and past temporal dependencies. We investigate our method on several public benchmarks for early intention prediction, improving the prediction performances at different anticipation times compared to the previous works.
National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
Convolutional Neural Networks have achieved impressive results in various tasks, but interpreting the internal mechanism is a challenging problem. To tackle this problem, we exploit a multi-channel attention mechanism in feature space. Our network architecture allows us to obtain an attention mask for each feature while existing CNN visualization methods provide only a common attention mask for all features. We apply the proposed multi-channel attention mechanism to multi-attribute recognition task. We can obtain different attention mask for each feature and for each attribute. Those analyses …
Can Segmentation Models Understand the World? Towards Proactive Affordance Reasoning via Visual Chain-of-Thought
Northwestern University（西北大学）；Northeastern University（东北大学）；South China University of Technology（华南理工大学）；Hong Kong Baptist University（香港 Baptist大学）；Beijing Normal - Hong Kong Baptist University（北京师范大学-香港 Baptist大学）
Scene Graph-guided SegCaptioning Transformer with Fine-grained Alignment for Controllable Video Segmentation and Captioning
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）；School of Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（机器人学院和机器人视觉感知与控制技术国家工程研究中心，湖南大学）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（计算机与人工智能学院，西南交通大学）；CRRC Zhuzhou Institute Company Ltd.（中车株洲院有限公司）；Nanyang Technological University（南洋理工大学）
Predicting vulnerable road user behavior is an essential prerequisite for deploying Automated Driving Systems (ADS) in the real-world. Pedestrian crossing intention should be recognized in real-time, especially for urban driving. Recent works have shown the potential of using vision-based deep neural network models for this task. However, these models are not robust and certain issues still need to be resolved. First, the global spatio-temproal context that accounts for the interaction between the target pedestrian and the scene has not been properly utilized. Second, the optimum strategy for …
Cross-modal Context-aware Learning for Visual Prompt Guided Multimodal Image Understanding in Remote Sensing
College of Computer Science and Electronic Engineering, Hunan University（计算机科学与电子工程学院，湖南大学）；School of Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（机器人学院及机器人视觉感知与控制技术国家工程研究中心，湖南大学）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（计算与人工智能学院，西南交通大学）
GTransPDM: A Graph-embedded Transformer with Positional Decoupling for Pedestrian Crossing Intention Prediction
Department of Traffic Information and Control Engineering, Jilin University（交通信息与控制工程系，吉林大学）；BIT-Barcelona Innovative Transportation Research Group, Civil Engineering School, UPC Barcelona Tech（巴塞罗那创新交通研究组，土木工程学院，UPC巴塞罗那技术学院）；Computer Vision Center (CVC), Computer Science Department, Universitat Autònoma de Barcelona (UAB)（计算机视觉中心（CVC），计算机科学系，巴塞罗那自治大学（UAB））
针对行人位置畸变及多模态时序建模难题，本文提出 GTransPDM，通过位置解耦与图嵌入 Transformer 融合姿态、位置和车辆运动特征，在 PIE 和 JAAD 上取得最优性能。
PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer
Department of Electrical Engineering and Information Technology, Intelligent Vehicles Lab (IVL), Munich University of Applied Science（电气工程与信息科技系，智能车辆实验室（IVL），慕尼黑应用科学大学）
EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videos
Fudan University（复旦大学）；The University of Tokyo（东京大学）；Zhejiang University（浙江大学）；Hong Kong University of Science and Technology（香港科技大学）；Nanjing University（南京大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
FreeEdit: Mask-free Reference-based Image Editing with Multi-modal Instruction
Chinese Academy of Sciences Institute of Information Engineering（中国科学院信息工程研究所）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Meituan（美团）
CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design
Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身人工智能研究所）；Shanghai Key Laboratory of Multimodal Embodied AI（上海市多模态具身人工智能重点实验室）；Bytedance Intelligent Creation（字节跳动智能创作）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）
How You Move Tells What You'll Do: Trajectory-Conditioned Egocentric Prediction
Khoury College of Computer Sciences, Northeastern University, Boston（北德文斯克学院，东北大学，波士顿）；Korea Advanced Institute of Science and Technology, Daejeon（韩国科学技术院，大田）；Department of Mathematics and Computer Science, University of Catania, Italy（卡塔尼亚大学数学与计算机科学系，意大利）
