When exploring the development of Artificial General Intelligence (AGI), a critical task for these models involves interpreting and processing information from multiple image inputs. However, Large Multimodal Models (LMMs) encounter two issues in such scenarios: (1) a lack of fine-grained perception, and (2) a tendency to blend information across multiple images. We first extensively investigate the capability of LMMs to perceive fine-grained visual details when dealing with multiple input images. The research focuses on two aspects: first, image-to-image matching (to evaluate whether LMMs can effectively reason and pair relevant images), and second, multi-image-to-text matching (to assess whether LMMs can accurately capture and summarize detailed image information). We conduct evaluations on a range of both open-source and closed-source large models, including GPT-4V, Gemini, OpenFlamingo, and MMICL. To enhance model performance, we further develop a Contrastive Chain-of-Thought (CoCoT) prompting approach based on multi-input multimodal models. This method requires LMMs to compare the similarities and differences among multiple image inputs, and then guide the models to answer detailed questions about multi-image inputs based on the identified similarities and differences. Our experimental results showcase CoCoT's proficiency in enhancing the multi-image comprehension capabilities of large multimodal models.
CRISP: Contrastive Residual Injection and Semantic Prompting for Continual Video Instance Segmentation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences（机器人与智能系统国家重点实验室，沈阳自动化研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Test-Time Multimodal Backdoor Detection by Contrastive Prompting
Chongqing University（重庆大学）；Nanyang Technological University（南洋理工大学）；Penn State University（宾夕法尼亚州立大学）；University of Melbourne（墨尔本大学）；Southeast University（东南大学）
From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation
Technical University of Munich（慕尼黑技术大学）；Korea University（韩国大学）；Kyung Hee University（庆熙大学）；Seoul National University of Science and Technology（首尔科学技术大学）
All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model
School of Computing, Australian National University（澳大利亚国立大学计算学院）；School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）
PromptGuard: An Orchestrated Prompting Framework for Principled Synthetic Text Generation for Vulnerable Populations using LLMs with Enhanced Safety, Fairness, and Controllability
Posts and Telecommunications Institute of Technology（邮电技术研究所）；Hanoi Architectural University（河内建筑大学）
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）；University of Central Florida（中佛罗里达大学）
Super Encoding Network: Recursive Association of Multi-Modal Encoders for Video Understanding
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；the School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Shanghai AI Laboratory（上海人工智能实验室）
"Humor, Art, or Misinformation?": A Multimodal Dataset for Intent-Aware Synthetic Image Detection
Department of Electrical & Computer Engineering, Aristotle University of Thessaloniki（电气与计算机工程系，阿基米德大学塞萨洛尼基分校）；Information Technology Institute, Centre for Research & Technology Hellas（信息科技研究所，希腊研究中心）
