多模态大模型 - arXivDaily 专题

2310.13292 2023-10-23 cs.CV cs.LG 57%

CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training

Kihyun You, Jawook Gu, Jiyeon Ham, Beomhee Park, Jiho Kim, Eun Kyoung Hong, Woonhyunk Baek, Byungseok Roh

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted by MICCAI 2023

2309.17319 2023-10-13 cs.AI 57%

Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models

Jinmeng Rao, Song Gao, Gengchen Mai, Krzysztof Janowicz

专题命中图文多模态：multimodal(abstract)；分类 cs.AI

Comments 1 figure

Journal ref ACM SIGSPATIAL 2023

2310.07394 2023-10-12 cs.CV 57%

CLIP for Lightweight Semantic Segmentation

Ke Jin, Wankou Yang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2310.06633 2023-10-11 cs.CV cs.CY 57%

Blind Dates: Examining the Expression of Temporality in Historical Photographs

Alexandra Barancová, Melvin Wevers, Nanne van Noord

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2310.03734 2023-10-06 cs.CV 57%

Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency

Tianhong Li, Sangnie Bhardwaj, Yonglong Tian, Han Zhang, Jarred Barber, Dina Katabi, Guillaume Lajoie, Huiwen Chang, Dilip Krishnan

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2310.02777 2023-10-05 cs.CL 57%

The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models

Chenwei Wu, Li Erran Li, Stefano Ermon, Patrick Haffner, Rong Ge, Zaiwei Zhang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CL

2310.02506 2023-10-05 cs.RO cs.AI 57%

Proactive Human-Robot Interaction using Visuo-Lingual Transformers

Pranay Mathur

专题命中图文多模态：multimodal(abstract)；分类 cs.AI

Comments Accepted to IROS'23 Workshop: Geriatronics: AI and Robotics for Health & Well-Being in Older Age and Workshop: Assistive Robotics for Citizens

2304.10592 2023-10-03 cs.CV 57%

MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, Mohamed Elhoseiny

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Project Website: https://minigpt-4.github.io/; Code, Pretrained Model, and Dataset: https://github.com/Vision-CAIR/MiniGPT-4; Deyao Zhu and Jun Chen contributed equally to this work

2309.16649 2023-09-29 cs.CV 57%

FLIP: Cross-domain Face Anti-spoofing with Language Guidance

Koushik Srivatsan, Muzammal Naseer, Karthik Nandakumar

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Accepted to ICCV-2023. Project Page: https://koushiksrivats.github.io/FLIP/

2309.16270 2023-09-29 cs.CL 57%

Social Media Fashion Knowledge Extraction as Captioning

Yifei Yuan, Wenxuan Zhang, Yang Deng, Wai Lam

专题命中图文多模态：multimodal(abstract)；分类 cs.CL

Comments Accepted by SIGIR-AP 2023

2309.15487 2023-09-28 cs.CV 57%

Tackling VQA with Pretrained Foundation Models without Further Training

Alvin De Jun Tan, Bingquan Shen

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2309.12314 2023-09-22 cs.CV 57%

TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance

Kan Wu, Houwen Peng, Zhenghong Zhou, Bin Xiao, Mengchen Liu, Lu Yuan, Hong Xuan, Michael Valenzuela, Xi, Chen, Xinggang Wang, Hongyang Chao, Han Hu

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted By ICCV 2023

2309.07372 2023-09-15 eess.AS cs.SD 57%

Training Audio Captioning Models without Audio

Soham Deshmukh, Benjamin Elizalde, Dimitra Emmanouilidou, Bhiksha Raj, Rita Singh, Huaming Wang

专题命中图文多模态：multimodal(abstract)；分类 eess.AS

2309.06517 2023-09-14 cs.CL 57%

Overview of Memotion 3: Sentiment and Emotion Analysis of Codemixed Hinglish Memes

Shreyash Mishra, S Suryavardan, Megha Chakraborty, Parth Patwa, Anku Rani, Aman Chadha, Aishwarya Reganti, Amitava Das, Amit Sheth, Manoj Chinnakotla, Asif Ekbal, Srijan Kumar

专题命中图文多模态：multi-modal(abstract)；分类 cs.CL

Comments Defactify2 @AAAI 2023

2306.01111 2023-09-14 cs.CV 57%

Exploring the Versatility of Zero-Shot CLIP for Interstitial Lung Disease Classification

Cara Van Uden, Christian Bluethgen, Maayane Attias, Malgorzata Polacin, Haiwei Henry Guo, Neha Simha, Rishi Raj, Curtis Langlotz

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 11 pages, 11 figures

2211.15398 2023-09-06 cs.CV cs.LG 57%

Leveraging per Image-Token Consistency for Vision-Language Pre-training

Yunhao Gou, Tom Ko, Hansi Yang, James Kwok, Yu Zhang, Mingxuan Wang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023

2308.16777 2023-09-04 cs.CV 57%

Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models

Minheng Ni, Yabo Zhang, Kailai Feng, Xiaoming Li, Yiwen Guo, Wangmeng Zuo

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.16689 2023-09-01 cs.CV 57%

ViLTA: Enhancing Vision-Language Pre-training through Textual Augmentation

Weihan Wang, Zhen Yang, Bin Xu, Juanzi Li, Yankui Sun

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 15 pages, 5 figures

2308.16493 2023-09-01 cs.AI cs.RO 57%

Expanding Frozen Vision-Language Models without Retraining: Towards Improved Robot Perception

Riley Tavassoli, Mani Amani, Reza Akhavian

专题命中图文多模态：multi-modal(abstract)；分类 cs.AI

Comments Preprint submitted to Information Fusion

2304.06028 2023-09-01 cs.CV 57%

RECLIP: Resource-efficient CLIP by Training with Small Images

Runze Li, Dahun Kim, Bir Bhanu, Weicheng Kuo

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Published at Transactions on Machine Learning Research

2308.14212 2023-08-29 cs.CV 57%

Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy

Sanoojan Baliah, Fadillah A. Maani, Santosh Sanjeev, Muhammad Haris Khan

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.12757 2023-08-25 cs.CV 57%

PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning

Mengya Han, Heliang Zheng, Chaoyue Wang, Yong Luo, Han Hu, Jing Zhang, Yonggang Wen

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2308.11605 2023-08-23 cs.CV 57%

GOPro: Generate and Optimize Prompts in CLIP using Self-Supervised Learning

Mainak Singha, Ankit Jha, Biplab Banerjee

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted at BMVC 2023

2306.00409 2023-08-23 cs.CV 57%

Adapting Pre-trained Language Models to Vision-Language Tasks via Dynamic Visual Prompting

Shubin Huang, Qiong Wu, Yiyi Zhou, Weijie Chen, Rongsheng Zhang, Xiaoshuai Sun, Rongrong Ji

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.10240 2023-08-22 cs.CV 57%

Generic Attention-model Explainability by Weighted Relevance Accumulation

Yiming Huang, Aozhe Jia, Xiaodan Zhang, Jiawei Zhang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.10061 2023-08-22 cs.CV 57%

DPL: Decoupled Prompt Learning for Vision-Language Models

Chen Xu, Yuhan Zhu, Guozhen Zhang, Haocheng Shen, Yixuan Liao, Xiaoxin Chen, Gangshan Wu, Limin Wang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments 11 pages, 5 figures, 8 tables

2308.08428 2023-08-21 cs.CV 57%

ALIP: Adaptive Language-Image Pre-training with Synthetic Caption

Kaicheng Yang, Jiankang Deng, Xiang An, Jiawei Li, Ziyong Feng, Jia Guo, Jing Yang, Tongliang Liu

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 15pages, 10figures, ICCV2023

2305.20087 2023-08-21 cs.CV 57%

Too Large; Data Reduction for Vision-Language Pre-Training

Alex Jinpeng Wang, Kevin Qinghong Lin, David Junhao Zhang, Stan Weixian Lei, Mike Zheng Shou

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments ICCV2023. Code: https://github.com/showlab/datacentric.vlp

2304.06712 2023-08-21 cs.CV 57%

What does CLIP know about a red circle? Visual prompt engineering for VLMs

Aleksandar Shtedritski, Christian Rupprecht, Andrea Vedaldi

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments ICCV 2023 Oral

2308.08881 2023-08-21 cs.CV 57%

Text-Only Training for Visual Storytelling

Yuechen Wang, Wengang Zhou, Zhenbo Lu, Houqiang Li

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments ACM MM 2023

1. 图文多模态 4672 篇

CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training

Building Privacy-Preserving and Secure Geospatial Artificial Intelligence Foundation Models

CLIP for Lightweight Semantic Segmentation

Blind Dates: Examining the Expression of Temporality in Historical Photographs

Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency

The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models

Proactive Human-Robot Interaction using Visuo-Lingual Transformers

MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

FLIP: Cross-domain Face Anti-spoofing with Language Guidance

Social Media Fashion Knowledge Extraction as Captioning

Tackling VQA with Pretrained Foundation Models without Further Training

TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance

Training Audio Captioning Models without Audio

Overview of Memotion 3: Sentiment and Emotion Analysis of Codemixed Hinglish Memes

Exploring the Versatility of Zero-Shot CLIP for Interstitial Lung Disease Classification

Leveraging per Image-Token Consistency for Vision-Language Pre-training

Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models

ViLTA: Enhancing Vision-Language Pre-training through Textual Augmentation

Expanding Frozen Vision-Language Models without Retraining: Towards Improved Robot Perception

RECLIP: Resource-efficient CLIP by Training with Small Images

Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy

PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning

GOPro: Generate and Optimize Prompts in CLIP using Self-Supervised Learning

Adapting Pre-trained Language Models to Vision-Language Tasks via Dynamic Visual Prompting

Generic Attention-model Explainability by Weighted Relevance Accumulation

DPL: Decoupled Prompt Learning for Vision-Language Models

ALIP: Adaptive Language-Image Pre-training with Synthetic Caption

Too Large; Data Reduction for Vision-Language Pre-Training

What does CLIP know about a red circle? Visual prompt engineering for VLMs

Text-Only Training for Visual Storytelling