多模态大模型 - arXivDaily 专题

2410.09733 2024-10-15 cs.CV 70%

MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models

Hang Hua, Yunlong Tang, Ziyun Zeng, Liangliang Cao, Zhengyuan Yang, Hangfeng He, Chenliang Xu, Jiebo Luo

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 21 pages, 15 figures

2405.14156 2024-10-08 cs.CV 70%

Unveiling the Tapestry of Consistency in Large Vision-Language Models

Yuan Zhang, Fei Xiao, Tao Huang, Chun-Kai Fan, Hongyuan Dong, Jiawen Li, Jiacong Wang, Kuan Cheng, Shanghang Zhang, Haoyuan Guo

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

2409.09582 2024-09-25 cs.CV 70%

NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training

Yiyi Tao, Zhuoyue Wang, Hang Zhang, Lun Wang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

2407.20171 2024-08-27 cs.CV 70%

Diffusion Feedback Helps CLIP See Better

Wenxuan Wang, Quan Sun, Fan Zhang, Yepeng Tang, Jing Liu, Xinlong Wang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2408.09530 2024-08-20 cs.AI 70%

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

Dawei Dai, Yuanhui Zhang, Long Xu, Qianlan Yang, Xiaojing Shen, Shuyin Xia, Guoyin Wang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.AI

Comments 8 pages, 4 figs

2407.18589 2024-07-29 cs.CV 70%

HICEScore: A Hierarchical Metric for Image Captioning Evaluation

Zequn Zeng, Jianqiao Sun, Hao Zhang, Tiansheng Wen, Yudi Su, Yan Xie, Zhengjue Wang, Bo Chen

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by ACM MM2024

2311.16254 2024-07-25 cs.CV cs.AI cs.CL cs.MM 70%

Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models

Samuele Poppi, Tobia Poppi, Federico Cocchi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ECCV 2024

2407.14177 2024-07-22 cs.CV 70%

EVLM: An Efficient Vision-Language Model for Visual Understanding

Kaibing Chen, Dong Shen, Hanwen Zhong, Huasong Zhong, Kui Xia, Di Xu, Wei Yuan, Yifei Hu, Bin Wen, Tianke Zhang, Changyi Liu, Dewen Fan, Huihui Xiao, Jiahong Wu, Fan Yang, Size Li, Di Zhang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

2403.12445 2024-07-16 cs.CV 70%

Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory

Sensen Gao, Xiaojun Jia, Xuhong Ren, Ivor Tsang, Qing Guo

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments ECCV2024. Code is available at https://github.com/SensenGao/VLPTransferAttack

2407.07403 2024-07-15 cs.CV 70%

A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Yu Cheng, Wei Hu

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

2403.18252 2024-06-18 cs.CV cs.AI cs.CL cs.LG cs.MM 70%

Beyond Embeddings: The Promise of Visual Table in Visual Reasoning

Yiwu Zhong, Zi-Yuan Hu, Michael R. Lyu, Liwei Wang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project page: https://github.com/LaVi-Lab/Visual-Table

2406.09356 2024-06-14 cs.CV eess.IV 70%

CMC-Bench: Towards a New Paradigm of Visual Signal Compression

Chunyi Li, Xiele Wu, Haoning Wu, Donghui Feng, Zicheng Zhang, Guo Lu, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, Weisi Lin

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2403.18715 2024-06-06 cs.CV cs.AI cs.CL cs.MM 70%

Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding

Xintong Wang, Jingheng Pan, Liang Ding, Chris Biemann

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to Findings of ACL 2024

2405.13127 2024-05-24 cs.CV cs.AI cs.CL cs.MM 70%

Towards Retrieval-Augmented Architectures for Image Captioning

Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Alessandro Nicolosi, Rita Cucchiara

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACM Transactions on Multimedia Computing, Communications and Applications (2024)

2405.10707 2024-05-22 cs.CV 70%

HARIS: Human-Like Attention for Reference Image Segmentation

Mengxi Zhang, Heqing Lian, Yiming Liu, Jie Chen

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

2312.07533 2024-05-20 cs.CV 70%

VILA: On Pre-training for Visual Language Models

Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, Song Han

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2024

2404.17845 2024-04-30 cs.CV 70%

Instance-free Text to Point Cloud Localization with Relative Position Awareness

Lichao Wang, Zhihao Yuan, Jinke Ren, Shuguang Cui, Zhen Li

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 12 pages, 10 figures, conference

2404.17779 2024-04-30 cs.CL 70%

Medical Vision-Language Pre-Training for Brain Abnormalities

Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CL

2404.09632 2024-04-16 cs.CV cs.LG 70%

Bridging Vision and Language Spaces with Assignment Prediction

Jungin Park, Jiyoung Lee, Kwanghoon Sohn

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments ICLR 2024 Camera-ready

2404.02132 2024-04-05 cs.CV 70%

ViTamin: Designing Scalable Vision Models in the Vision-Language Era

Jieneng Chen, Qihang Yu, Xiaohui Shen, Alan Yuille, Liang-Chieh Chen

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2024; https://github.com/Beckschen/ViTamin

2403.17007 2024-03-26 cs.CV 70%

DreamLIP: Language-Image Pre-training with Long Captions

Kecheng Zheng, Yifei Zhang, Wei Wu, Fan Lu, Shuailei Ma, Xin Jin, Wei Chen, Yujun Shen

专题命中图文多模态：MLLM(abstract)；image-text(abstract)；分类 cs.CV

2312.02520 2024-03-19 cs.CV 70%

Towards More Unified In-context Visual Understanding

Dianmo Sheng, Dongdong Chen, Zhentao Tan, Qiankun Liu, Qi Chu, Jianmin Bao, Tao Gong, Bin Liu, Shengwei Xu, Nenghai Yu

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2403.06136 2024-03-12 cs.CV 70%

RESTORE: Towards Feature Shift for Vision-Language Prompt Learning

Yuncheng Yang, Chuyan Zhang, Zuopeng Yang, Yuting Gao, Yulei Qin, Ke Li, Xing Sun, Jie Yang, Yun Gu

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 18 pages, 5 figures

2403.02581 2024-03-06 cs.CV cs.SE 70%

VEglue: Testing Visual Entailment Systems via Object-Aligned Joint Erasing

Zhiyuan Chang, Mingyang Li, Junjie Wang, Cheng Li, Qing Wang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments 12pages, 3 figures

2402.19479 2024-03-01 cs.CV 70%

Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers

Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, Sergey Tulyakov

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2024. Project Page: https://snap-research.github.io/Panda-70M

2402.13577 2024-02-22 cs.CL 70%

BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models

Xueliang Zhao, Xinting Huang, Tingchen Fu, Qintong Li, Shansan Gong, Lemao Liu, Wei Bi, Lingpeng Kong

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CL

Comments Preprint

2305.11481 2024-02-15 cs.CV 70%

CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation

Wenxuan Wang, Jing Liu, Xingjian He, Yisi Zhang, Chen Chen, Jiachen Shen, Yan Zhang, Jiangyun Li

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2402.06198 2024-02-14 cs.CV 70%

GS-CLIP: Gaussian Splatting for Contrastive Language-Image-3D Pretraining from Real-World Data

Haoyuan Li, Yanpeng Zhou, Yihan Zeng, Hang Xu, Xiaodan Liang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments The content of the technical report needs to be updated and retracted to avoid other impacts

2401.02814 2024-02-02 cs.RO cs.CV 70%

Object-Centric Instruction Augmentation for Robotic Manipulation

Junjie Wen, Yichen Zhu, Minjie Zhu, Jinming Li, Zhiyuan Xu, Zhengping Che, Chaomin Shen, Yaxin Peng, Dong Liu, Feifei Feng, Jian Tang

专题命中图文多模态：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments accepted to ICRA2024

2305.14800 2024-01-24 cs.CV 70%

Exploring Diverse In-Context Configurations for Image Captioning

Xu Yang, Yongliang Wu, Mingzhuo Yang, Haokun Chen, Xin Geng

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by NeurIPS2023

1. 图文多模态 4672 篇

MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models

Unveiling the Tapestry of Consistency in Large Vision-Language Models

NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training

Diffusion Feedback Helps CLIP See Better

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

HICEScore: A Hierarchical Metric for Image Captioning Evaluation

Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models

EVLM: An Efficient Vision-Language Model for Visual Understanding

Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory

A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

Beyond Embeddings: The Promise of Visual Table in Visual Reasoning

CMC-Bench: Towards a New Paradigm of Visual Signal Compression

Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding

Towards Retrieval-Augmented Architectures for Image Captioning

HARIS: Human-Like Attention for Reference Image Segmentation

VILA: On Pre-training for Visual Language Models

Instance-free Text to Point Cloud Localization with Relative Position Awareness

Medical Vision-Language Pre-Training for Brain Abnormalities

Bridging Vision and Language Spaces with Assignment Prediction

ViTamin: Designing Scalable Vision Models in the Vision-Language Era

DreamLIP: Language-Image Pre-training with Long Captions

Towards More Unified In-context Visual Understanding

RESTORE: Towards Feature Shift for Vision-Language Prompt Learning

VEglue: Testing Visual Entailment Systems via Object-Aligned Joint Erasing

Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers

BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models

CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation

GS-CLIP: Gaussian Splatting for Contrastive Language-Image-3D Pretraining from Real-World Data

Object-Centric Instruction Augmentation for Robotic Manipulation

Exploring Diverse In-Context Configurations for Image Captioning