Detecting hidden or partially concealed objects remains a fundamental challenge in multimodal environments, where factors like occlusion, camouflage, and lighting variations significantly hinder performance. Traditional RGB-based detection methods often fail under such adverse conditions, motivating the need for more robust, modality-agnostic approaches. In this work, we present HiddenObject, a fusion framework that integrates RGB, thermal, and depth data using a Mamba-based fusion mechanism. Our method captures complementary signals across modalities, enabling enhanced detection of obscured or camouflaged targets. Specifically, the proposed approach identifies modality-specific features and fuses them in a unified representation that generalizes well across challenging scenarios. We validate HiddenObject across multiple benchmark datasets, demonstrating state-of-the-art or competitive performance compared to existing methods. These results highlight the efficacy of our fusion design and expose key limitations in current unimodal and naïve fusion strategies. More broadly, our findings suggest that Mamba-based fusion architectures can significantly advance the field of multimodal object detection, especially under visually degraded or complex conditions.
Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection
Information Science and Technology College, Dalian Maritime University（大连海事大学信息科学与技术学院）；Beijing University of Technology（北京理工大学）；Dalian University of Technology（大连理工大学）
Deep neural networks (DNNs) usually fail to generalize well to outside of distribution (OOD) data, especially in the extreme case of single domain generalization (single-DG) that transfers DNNs from single domain to multiple unseen domains. Existing single-DG techniques commonly devise various data-augmentation algorithms, and remould the multi-source domain generalization methodology to learn domain-generalized (semantic) features. Nevertheless, these methods are typically modality-specific, thereby being only applicable to one single modality (e.g., image). In contrast, we target a versatile…
Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS
Computational Biomedicine Laboratory of the Foundation for Research and Technology (FORTH)（希腊国家研究与技术基金会计算生物医学实验室）；Department of Electrical & Computer Engineering Hellenic Mediterranean University（希腊地中海大学电气与计算机工程系）
BiXFormer: A Robust Framework for Maximizing Modality Effectiveness in Multi-Modal Semantic Segmentation
Graduate School of Informatics, Nagoya University（名古屋大学信息学研究科）；AI Thrust, The Hong Kong University of Science and Technology（香港科学与技术大学人工智能 thrust）；INSAIT, Sofia University（索菲亚大学INSAIT）
Heterogeneous face recognition is a challenging task due to the large modality discrepancy and insufficient cross-modal samples. Most existing works focus on discriminative feature transformation, metric learning and cross-modal face synthesis. However, the fact that cross-modal faces are always coupled by domain (modality) and identity information has received little attention. Therefore, how to learn and utilize the domain-private feature and domain-agnostic feature for modality adaptive face recognition is the focus of this work. Specifically, this paper proposes a Feature Aggregation Netwo…
Modality-Agnostic Input Channels Enable Segmentation of Brain lesions in Multimodal MRI with Sequences Unavailable During Training
Department of Engineering Science, University of Oxford, Oxford, UK（工程科学系，牛津大学，牛津，英国）；Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK（牛津大学临床神经科学系，牛津，英国）
FreeSliders: Training-Free, Modality-Agnostic Concept Sliders for Fine-Grained Diffusion Control in Images, Audio, and Video
Faculty of Computer and Information Science, Ben-Gurion University of the Negev（计算机与信息科学学院，贝内-约尔大学纳格尔分校）；Lightricks（Lightricks公司）；School of Electrical and Computer Engineering, Ben-Gurion University of the Negev（电气与计算机工程学院，贝内-约尔大学纳格尔分校）
Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration
DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Hupan Lab（湖畔实验室）；College of Computer Science and Technology, Zhejiang University, China（浙江大学计算机科学与技术学院）；Shengjing Hospital of China Medical University, China（中国医科大学附属盛京医院）
Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations
Academy for Engineering and Technology, Fudan University（复旦大学工程技术研究院）；Digital Medical Research Center, School of Basic Medical Science, Fudan University（复旦大学基础医学院数字医学研究中心）；University of South Carolina（南卡罗来纳大学）
Fine-Grained Scene Image Classification with Modality-Agnostic Adapter
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Innovation Center of Calligraphy and Painting Creation Technology, MCT（中国传媒书画创作技术创新中心）；Videt Technology（维德科技）
Image retrieval with natural language feedback offers the promise of catalog search based on fine-grained visual features that go beyond objects and binary attributes, facilitating real-world applications such as e-commerce. Our Modality-Agnostic Attention Fusion (MAAF) model combines image and text features and outperforms existing approaches on two visual search with modifying phrase datasets, Fashion IQ and CSS, and performs competitively on a dataset with only single-word modifications, Fashion200k. We also introduce two new challenging benchmarks adapted from Birds-to-Words and Spot-the-D…
UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation
LIGM, Ecole Nationale des Ponts et Chaussées, IP Paris, Univ Gustave Eiffel, CNRS（LIGM，巴黎国立桥梁与道路学院，IP巴黎，埃夫尔大学，CNRS）；LASTIG, Univ Gustave Eiffel, IGN, ENSG（LASTIG，埃夫尔大学，国家地理信息学院，ENSG）；IGN（国家地理信息学院）；CNES（国家航天中心）；EFEO（埃克塞特东方研究院）
Modality-Agnostic Style Transfer for Holistic Feature Imputation
Pohang University of Science and Technology（浦项科技大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of North Carolina at Greensboro（北卡罗来纳大学格林斯伯勒分校）
XTransfer: Modality-Agnostic Few-Shot Model Transfer for Human Sensing at the Edge
Macquarie University, Sydney, NSW, Australia（麦考瑞大学，悉尼，新南威尔士州，澳大利亚）；Nanyang Technological University, Singapore（南洋理工大学，新加坡）；The University of Auckland, Auckland, New Zealand（奥克兰大学，奥克兰，新西兰）
Mind the Gap: Learning Modality-Agnostic Representations with a Cross-Modality UNet
Tianjin Key Laboratory of Intelligent Robotics, College of Artificial Intelligence, Nankai University, China（天津智能机器人重点实验室，人工智能学院，南开大学，中国）；Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, Nankai University, China（可信行为智能工程研究中心，教育部，南开大学，中国）；Department of Computer Science, Haifa University, Israel（计算机科学系，海法大学，以色列）；VisionMetric Ltd, Canterbury, Kent, UK（VisionMetric Ltd，坎特伯雷，肯特，英国）
PelviNeXt: A Modality-Agnostic Hybrid Network for Pelvic Imaging in Women's Health
Center for Computational & Data Sciences, Independent University, Bangladesh（孟加拉国独立大学计算与数据科学中心）；Department of Computer Science and Engineering, Independent University, Bangladesh（孟加拉国独立大学计算机科学与工程系）
Detecting Dataset Bias in Medical AI: A Generalized and Modality-Agnostic Auditing Framework
The Johns Hopkins University（约翰霍普金斯大学）；The Johns Hopkins University Applied Physics Laboratory（约翰霍普金斯大学应用物理实验室）；Center for Devices and Radiological Health, U.S. Food and Drug Administration（美国食品药品监督管理局设备与放射健康中心）
Department of Biomedical Engineering, Yale University（耶鲁大学生物医学工程系）；Department of Radiology & Biomedical Imaging, Yale School of Medicine（耶鲁大学医学院放射科与生物医学成像系）；Department of Electrical Engineering, Yale University（耶鲁大学电气工程系）；Department of Statistics & Data Science, Yale University（耶鲁大学统计与数据科学系）；Yale School of Medicine（耶鲁大学医学院）
Domain Adaptation (DA) has the potential to greatly help the generalization of deep learning models. However, the current literature usually assumes to transfer the knowledge from the source domain to a specific known target domain. Domain Agnostic Learning (DAL) proposes a new task of transferring knowledge from the source domain to data from multiple heterogeneous target domains. In this work, we propose the Domain-Agnostic Learning framework with Anatomy-Consistent Embedding (DALACE) that works on both domain-transfer and task-transfer to learn a disentangled representation, aiming to not o…
Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment
School of Computer Science, University of Sydney（悉尼大学计算机学院）；School of Computing, Engineering and Mathematical Sciences, La Trobe University（拉筹伯大学计算、工程与数学科学学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；The Chinese University of Hong Kong（香港中文大学）；Lehigh University（理海大学）；Harvard University（哈佛大学）；The University of Georgia（佐治亚大学）
Zhejiang University-University of Illinois at Urbana-Champaign Institute（浙江大学-伊利诺伊大学厄巴纳香槟分校联合学院）；Zhejiang University（浙江大学）
Multimodal sentiment analysis (MSA) is an important way of observing mental activities with the help of data captured from multiple modalities. However, due to the recording or transmission error, some modalities may include incomplete data. Most existing works that address missing modalities usually assume a particular modality is completely missing and seldom consider a mixture of missing across multiple modalities. In this paper, we propose a simple yet effective meta-sampling approach for multimodal sentiment analysis with missing modalities, namely Missing Modality-based Meta Sampling (M3…
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；MIT BCS（麻省理工学院脑与认知科学系）；MIT CBMM（麻省理工学院脑、心智与机器中心）
Deep neural networks have been used widely to learn the latent structure of datasets, across modalities such as images, shapes, and audio signals. However, existing models are generally modality-dependent, requiring custom architectures and objectives to process different classes of signals. We leverage neural fields to capture the underlying structure in image, shape, audio and cross-modal audiovisual domains in a modality-independent manner. We cast our task as one of learning a manifold, where we aim to infer a low-dimensional, locally linear subspace in which our data resides. By enforcing…
Rutgers University（罗格斯大学）；SenseBrain Research（SenseBrain研究院）；National Heart Centre Singapore（新加坡国家心脏中心）；Duke-National University Of Singapore（杜克-新加坡国立大学）
Multi-modality images have been widely used and provide comprehensive information for medical image analysis. However, acquiring all modalities among all institutes is costly and often impossible in clinical settings. To leverage more comprehensive multi-modality information, we propose a privacy secured decentralized multi-modality adaptive learning architecture named ModalityBank. Our method could learn a set of effective domain-specific modulation parameters plugged into a common domain-agnostic network. We demonstrate by switching different sets of configurations, the generator could outpu…
