Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution
Shanghai Engineering Research Center of AI & Robotics(上海市AI与机器人工程研究中心);Academy for Engineering & Technology, Fudan University(复旦大学工程技术研究院);East China University of Science and Technology(华东理工大学);College of Information Technology, Shanghai Ocean University(上海海洋大学信息技术学院)
Junxiong Lin, Yan Wang, Zeng Tao, Boyang Wang, Qing Zhao, Haorang Wang, Xuan Tong, Xinji Mai, Yuxuan Lin, Wei Song, Jiawen Yu, Shaoqi Yan, Wenqiang Zhang
Pre-trained diffusion models utilized for image generation encapsulate a substantial reservoir of a priori knowledge pertaining to intricate textures. Harnessing the potential of leveraging this a priori knowledge in the context of image super-resolution presents a compelling avenue. Nonetheless, prevailing diffusion-based methodologies presently overlook the constraints imposed by degradation information on the diffusion process. Furthermore, these methods fail to consider the spatial variability inherent in the estimated blur kernel, stemming from factors such as motion jitter and out-of-focus elements in open-environment scenarios. This oversight results in a notable deviation of the image super-resolution effect from fundamental realities. To address these concerns, we introduce a framework known as Adaptive Multi-modal Fusion of \textbf{S}patially Variant Kernel Refinement with Diffusion Model for Blind Image \textbf{S}uper-\textbf{R}esolution (SSR). Within the SSR framework, we propose a Spatially Variant Kernel Refinement (SVKR) module. SVKR estimates a Depth-Informed Kernel, which takes the depth information into account and is spatially variant. Additionally, SVKR enhance the accuracy of depth information acquired from LR images, allowing for mutual enhancement between the depth map and blur kernel estimates. Finally, we introduce the Adaptive Multi-Modal Fusion (AMF) module to align the information from three modalities: low-resolution images, depth maps, and blur kernels. This alignment can constrain the diffusion model to generate more authentic SR results.
Pattern-Aware Diffusion Synthesis of fMRI/dMRI with Tissue and Microstructural Refinement
Department of Computer Science and Technology, Harbin Institute of Technology(计算机科学与技术系,哈尔滨工业大学);School of Intelligence Science and Engineering, College of Artificial Intelligence, Harbin Institute of Technology(智能科学与工程学院,人工智能学院,哈尔滨工业大学);School of Biomedical Engineering, National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical, Measurements and Ultrasound Imaging, Shenzhen University Medical School, Shenzhen University, Shenzhen, China(生物医学工程学院,医学超声关键技术工程实验室,广东生物医学、测量与超声成像重点实验室,深圳大学医学院,深圳大学,深圳,中国);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究院,中国科学院);Department of Radiology, The People’s Hospital of Guangxi Zhuang Autonomous Region, Guangxi Academy of Medical Sciences(放射科,广西壮族自治区人民医院,广西医学院)
Multi-modal Fusion and Query Refinement Network for Video Moment Retrieval and Highlight Detection
School of Electronic Science and Engineering, Nanjing University(南京大学电子科学与工程学院);School of Physics and Electronics, Hubei Normal University(湖北师范大学物理与电子科学学院);Gosuncn Chuanglian Technology Co., Ltd.(高新兴创联科技有限公司)
CSDN: Cross-modal Shape-transfer Dual-refinement Network for Point Cloud Completion
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Delft University of Technology(代尔夫特理工大学);Lingnan University(岭南大学);The Hong Kong Polytechnic University(香港理工大学)
Point-aware Interaction and CNN-induced Refinement Network for RGB-D Salient Object Detection
Beijing Jiaotong University(北京交通大学);Shandong University(山东大学);Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学研究所);Southern University of Science and Technology(南方科技大学);City University of Hong Kong(香港城市大学)
Towards Multi-modality Fusion and Prototype-based Feature Refinement for Clinically Significant Prostate Cancer Classification in Transrectal Ultrasound
School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University(深圳大学医学部生物医学工程学院);Sun Yat-sen University Cancer Center(中山大学肿瘤防治中心)
DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Department of Computer Science, TU Darmstadt(达姆施塔特工业大学计算机科学系)
MLCR: Multi-Level Cue Refinement for Long-Term Multimodal Action Quality Assessment
Qiushi Academy for Advanced Studies (QAAS), Zhejiang University(浙江大学启斯特先进研究院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院);State Key Lab of Brain-Machine Intelligence(脑机智能国家重点实验室);Collaborative Innovation Center for Artificial Intelligence by MOE and Zhejiang Provincial Government (ZJU)(教育部-浙江省人工智能协同创新中心);Zhejiang Lab(浙江实验室)
A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction
Tandon School of Engineering, New York University(纽约大学坦登工程学院);NYU Langone Health, New York University(纽约大学朗格尼健康中心);New York University Abu Dhabi(纽约大学阿布扎比分校)
Multi-modal Instance Refinement for Cross-domain Action Recognition
University of Electronic Science and Technology of China(电子科技大学);Shenzhen Institute for Advanced Study(深圳高等研究院);China Unicom Shenzhen Branch(中国联通深圳分公司)
Multi-Modal Prompt Learning on Blind Image Quality Assessment
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(厦门大学多媒体可信感知与高效计算教育部重点实验室);Tencent Youtu Lab(腾讯优图实验室);School of Information and Electronics, Beijing Institute of Technology(北京理工大学信息与电子学院);School of Computer Science and Technology, Ocean University of China(中国海洋大学计算机科学与技术学院)
Multi-scale Decomposed Convolution Refinement Network for Visible-Infrared Person Re-Identification
School of Computer Science and Technology, Xinjiang University(新疆大学计算机科学与技术学院);Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University(新疆大学丝绸之路多语言认知计算联合国际研究实验室)
CrossTracker: Robust Multi-modal 3D Multi-Object Tracking via Cross Correction
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);Collaborative Innovation Center of Novel Software Technology and Industrialization(新型软件技术与产业化协同创新中心);School of Science and Technology, Hong Kong Metropolitan University(香港都会大学科技学院);School of Computer Science, Jiangsu University of Technology(江苏理工学院计算机学院);Delft University of Technology(代尔夫特理工大学)
IT-TextFusion: Iterative Text-Image Interaction with Text-Guided Residual Refinement for Degradation-Aware Image Fusion
School of Automation, Southeast University(东南大学自动化学院);Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich(慕尼黑工业大学机器人、人工智能与实时系统研究所)
ASFR-Net: Adversarial Alignment and Spatio-Frequency Refinement Network for Heterogeneous Remote Sensing Image Change Detection
MOE Key Laboratory of ICSP, IMIS Laboratory of Anhui Province, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, Zenmorn-AHU AI Joint Laboratory, School of Computer Science and Technology, Anhui University(教育部智能计算与信号处理重点实验室、安徽省智能感知与机器智能重点实验室、安徽省多模态认知计算重点实验室、安徽大学智源-安徽大学人工智能联合实验室、安徽大学计算机科学与技术学院);School of Public Safety and Emergency Management, Anhui University of Science and Technology(安徽理工大学公共安全与应急管理学院)