Recently, the advent of Large Visual-Language Models (LVLMs) has received increasing attention across various domains, particularly in the field of visual document understanding (VDU). Different from conventional vision-language tasks, VDU is specifically concerned with text-rich scenarios containing abundant document elements. Nevertheless, the importance of fine-grained features remains largely unexplored within the community of LVLMs, leading to suboptimal performance in text-rich scenarios. In this paper, we abbreviate it as the fine-grained feature collapse issue. With the aim of filling this gap, we propose a contrastive learning framework, termed Document Object COntrastive learning (DoCo), specifically tailored for the downstream tasks of VDU. DoCo leverages an auxiliary multimodal encoder to obtain the features of document objects and align them to the visual features generated by the vision encoder of LVLM, which enhances visual representation in text-rich scenarios. It can represent that the contrastive learning between the visual holistic representations and the multimodal fine-grained features of document objects can assist the vision encoder in acquiring more effective visual cues, thereby enhancing the comprehension of text-rich documents in LVLMs. We also demonstrate that the proposed DoCo serves as a plug-and-play pre-training method, which can be employed in the pre-training of various LVLMs without inducing any increase in computational complexity during the inference process. Extensive experimental results on multiple benchmarks of VDU reveal that LVLMs equipped with our proposed DoCo can achieve superior performance and mitigate the gap between VDU and generic vision-language tasks.
Enhancing Multi-view Stereo with Contrastive Matching and Weighted Focal Loss
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Research Institute of Tsinghua, Pearl River Delta（清华大学珠三角研究院）
近年来，基于学习的多视图立体视觉（MVS）方法取得了令人瞩目的进展，并超越了传统方法。然而，其准确性和完整性仍在努力。在本文中，我们提出了一种新的方法，以提高现有的网络的性能启发对比学习和特征匹配。首先，提出了一种对比度匹配损失（CML）算法，将深度维上正确匹配的点作为正样本，其他点作为负样本，根据特征的相似性计算对比度损失。我们进一步提出了一个加权焦点损失（WFL）更好的分类能力，这削弱了低置信度像素在不重要的地区的损失，根据预测的信心的贡献。在DTU，Tanks and Temples和BlendedMVS数据集上进行的大量实验表明，我们的方法实现了最先进的性能，并在基线网络上有显着的改进。
ContrastDiagnosis: Enhancing Interpretability in Lung Nodule Diagnosis Using Contrastive Learning
East China Normal University（华东师范大学）；NVIDIA Corp.（英伟达公司）；Nagoya University（名古屋大学）；National Institute of Informatics（信息学研究所）；The First Affiliated Hospital of Nanjing Medical University（南京医科大学第一附属医院）
Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning
Paul C. Lauterbur Research Center for Biomedical Imaging（保罗·C·劳特伯生物医学成像研究中心）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）；Pengcheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Harvard University（哈佛大学）；Shanghai AI Lab（上海人工智能实验室）
Think Bright, Diffuse Nice: Enhancing T2I-ICL via Inductive-Bias Hint Instruction and Query Contrastive Decoding
Cao Tu Li (Guangzhou) Technology Co., Ltd（曹图利（广州）科技有限公司）；South China University of Technology（华南理工大学）；Washington State University（华盛顿州立大学）；Hong Kong Baptist University（香港 Baptist 大学）
SynDoc: A Hybrid Discriminative-Generative Framework for Enhancing Synthetic Domain-Adaptive Document Key Information Extraction
The University of Western Australia（西澳大学）；The University of Melbourne（墨尔本大学）；The University of Sydney（悉尼大学）；Weill Cornell Medicine, Cornell University（韦尔·科恩医学中心，康奈尔大学）
Enhancing Whole Slide Image Classification through Supervised Contrastive Domain Adaptation
HUMAN-tech；Universidad Politécnica de Valencia (UPV)（瓦伦西亚理工大学）；Valencian Graduate School and Research Network for Artificial Intelligence (valgrAI)（瓦伦西亚人工智能研究生学院及研究网络）
Enhancing IMU-Based Online Handwriting Recognition via Contrastive Learning with Zero Inference Overhead
Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-艾萨克-亚历山大大学埃尔朗根-纽伦堡）；STABILO International GmbH（STABILO国际有限公司）；Ludwig-Maximilians-Universität München（路德维希-马克西米利安大学慕尼黑）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Helmholtz Zentrum München - German Research Center for Environmental Health（海德堡中心慕尼黑 - 德国环境健康研究中心）
Enhancing medical vision-language contrastive learning via inter-matching relation modelling
the University of Sydney（悉尼大学）；Royal Prince Alfred Hospital（皇家阿尔弗雷德王子医院）；Shanghai Jiao Tong University（上海交通大学）；Institute of Translational Medicine（转化医学研究院）
Enhancing Multimodal Medical Image Classification using Cross-Graph Modal Contrastive Learning
Department of Systems Engineering, Stevens Institute of Technology, Hoboken, New Jersey, USA（系统工程系，史蒂文斯理工学院）；Department of Nuclear Medicine, Kaohsiung Chang Gung Memorial Hospital, Kaohsiung, Taiwan（高雄长庚纪念医院核医学部）；Institute of Statistics, National University of Kaohsiung, Kaohsiung, Taiwan（国立高雄大学统计研究所）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China（深圳先进技术研究院，中国科学院）
Enhancing Contrastive Learning for Geolocalization by Discovering Hard Negatives on Semivariograms
Professorship Big Geospatial Data Management, Technical University of Munich Germany（技术大学慕尼黑大数据地理空间管理教授职位）；University of Maine United States（缅因大学）；University of the Bundeswehr Munich Germany（联邦国防军大学慕尼黑）
