多模态大模型

CXR-Agent: Vision-language models for chest X-ray interpretation with uncertainty aware radiology reporting

Naman Sharma

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Supervised by Professor Ben Glocker

Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks

Tingyu Qu, Tinne Tuytelaars, Marie-Francine Moens

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted at ECCV 2024

AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization

Shixiong Xu, Chenghao Zhang, Lubin Fan, Gaofeng Meng, Shiming Xiang, Jieping Ye

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted at ECCV 2024

Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributes

Yusuke Hirota, Jerone T. A. Andrews, Dora Zhao, Orestis Papakyriakopoulos, Apostolos Modas, Yuta Nakashima, Alice Xiang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP

Zixiang Chen, Yihe Deng, Yuanzhi Li, Quanquan Gu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.AI

Comments 31 pages, 7 tables, 6 figures. In ICLR 2024

Towards a text-based quantitative and explainable histopathology image analysis

Anh Tien Nguyen, Trinh Thi Le Vuong, Jin Tae Kwak

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments MICCAI 2024 - Early acceptance (Top 11%)

Does CLIP Know My Face?

Dominik Hintersdorf, Lukas Struppek, Manuel Brack, Felix Friedrich, Patrick Schramowski, Kristian Kersting

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Published in the Journal of Artificial Intelligence Research (JAIR)

Journal ref Journal of Artificial Intelligence Research (JAIR), Vol. 80 (2024)

Benchmarking and Improving Detail Image Caption

Hongyuan Dong, Jiawen Li, Bohong Wu, Jiacong Wang, Yuan Zhang, Haoyuan Guo

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Smart Vision-Language Reasoners

Denisa Roberts, Lucas Roberts

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.AI

Comments Accepted in ICML 2024 MATH AI Workshop

CLIP-DR: Textual Knowledge-Guided Diabetic Retinopathy Grading with Ranking-aware Prompting

Qinkai Yu, Jianyang Xie, Anh Nguyen, He Zhao, Jiong Zhang, Huazhu Fu, Yitian Zhao, Yalin Zheng, Yanda Meng

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by MICCAI 2024

Multi-Peptide: Multimodality Leveraged Language-Graph Learning of Peptide Properties

Srivathsan Badrinarayanan, Chakradhar Guntuboina, Parisa Mollaei, Amir Barati Farimani

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.AI

The Solution for the CVPR2023 NICE Image Captioning Challenge

Xiangyu Wu, Yi Gao, Hailiang Zhang, Yang Yang, Weili Guo, Jianfeng Lu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learning

Ji Ma, Wei Suo, Peng Wang, Yanning Zhang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by IJCAI-24

Learning Robust 3D Representation from CLIP via Dual Denoising

Shuqing Luo, Bowen Qu, Wei Gao

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Distilling Knowledge from Text-to-Image Generative Models Improves Visio-Linguistic Reasoning in CLIP

Samyadeep Basu, Shell Xu Hu, Maziar Sanjabi, Daniela Massiceti, Soheil Feizi

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Short paper

Revisiting Vision-Language Features Adaptation and Inconsistency for Social Media Popularity Prediction

Chih-Chung Hsu, Chia-Ming Lee, Yu-Fan Lin, Yi-Shiuan Chou, Chih-Yu Jian, Chi-Han Tsai

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.MM

Comments Submission of the 7th Social Media Prediction Challenge

Assistive Image Annotation Systems with Deep Learning and Natural Language Capabilities: A Review

Moseli Mots'oehli

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted IEEE ETNCC 2024, 9 pages

LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models

Gabriela Ben Melech Stan, Estelle Aflalo, Raanan Yehezkel Rohekar, Anahita Bhiwandiwalla, Shao-Yen Tseng, Matthew Lyle Olson, Yaniv Gurwicz, Chenfei Wu, Nan Duan, Vasudev Lal

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Retrieval-Augmented Egocentric Video Captioning

Jilan Xu, Yifei Huang, Junlin Hou, Guo Chen, Yuejie Zhang, Rui Feng, Weidi Xie

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments CVPR 2024. Project page is available at: https://jazzcharles.github.io/Egoinstructor/

Disturbing Image Detection Using LMM-Elicited Emotion Embeddings

Maria Tzelepi, Vasileios Mezaris

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted for publication, LVLM Workshop @ IEEE Int. Conf. on Image Processing (ICIP 2024), Abu Dhabi, United Arab Emirates, Oct. 2024. This is the authors' "accepted version"

The Solution for CVPR2024 Foundational Few-Shot Object Detection Challenge

Hongpeng Pan, Shifeng Yi, Shouwei Yang, Lei Qi, Bing Hu, Yi Xu, Yang Yang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments CVPR2024 Foundational Few-Shot Object Detection Challenge

A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions

Jack Urbanek, Florian Bordes, Pietro Astolfi, Mary Williamson, Vasu Sharma, Adriana Romero-Soriano

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Leveraging VLM-Based Pipelines to Annotate 3D Objects

Rishabh Kabra, Loic Matthey, Alexander Lerchner, Niloy J. Mitra

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Precision Empowers, Excess Distracts: Visual Question Answering With Dynamically Infused Knowledge In Language Models

Manas Jhalani, Annervaz K M, Pushpak Bhattacharyya

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL

Comments 16 pages, 12 figures

Comparison Visual Instruction Tuning

Wei Lin, Muhammad Jehanzeb Mirza, Sivan Doveh, Rogerio Feris, Raja Giryes, Sepp Hochreiter, Leonid Karlinsky

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Project page: https://wlin-at.github.io/cad_vi ; Huggingface dataset repo: https://huggingface.co/datasets/wlin21at/CaD-Inst

CLIP-Driven Cloth-Agnostic Feature Learning for Cloth-Changing Person Re-Identification

Shuang Li, Jiaxu Leng, Guozhang Li, Ji Gan, Haosheng chen, Xinbo Gao

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

LM4LV: A Frozen Large Language Model for Low-level Vision Tasks

Boyang Zheng, Jinjin Gu, Shijun Li, Chao Dong

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyond

Pengyuan Lyu, Yulin Li, Hao Zhou, Weihong Ma, Xingyu Wan, Qunyi Xie, Liang Wu, Chengquan Zhang, Kun Yao, Errui Ding, Jingdong Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization

Yunpeng Zhao, Cheng Chen, Qing You Pang, Quanzheng Li, Carol Tang, Beng-Ti Ang, Yueming Jin

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter

Junfei Xiao, Zheng Xu, Alan Yuille, Shen Yan, Boyu Wang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Technical report, 15 pages; v2 fix typos, add additional results in appendix