多模态大模型 - arXivDaily 专题

2306.08330 2023-09-12 cs.CV 79%

Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction

Yingxue Xu, Hao Chen

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments 11 pages, 4 figures, accepted by ICCV 2023

2309.01262 2023-09-06 cs.CV cs.HC cs.LG eess.SP 79%

Multimodal Contrastive Learning with Hard Negative Sampling for Human Activity Recognition

Hyeongju Choi, Apoorva Beedu, Irfan Essa

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2308.16649 2023-09-01 cs.CV 79%

Learning with Multi-modal Gradient Attention for Explainable Composed Image Retrieval

Prateksha Udhayanan, Srikrishna Karanam, Balaji Vasan Srinivasan

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

2308.14613 2023-08-29 cs.CV 79%

MS-Net: A Multi-modal Self-supervised Network for Fine-Grained Classification of Aircraft in SAR Images

Bingying Yue, Jianhao Li, Hao Shi, Yupei Wang, Honghu Zhong

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

2306.08306 2023-08-22 cs.MM cs.LG 79%

Towards Balanced Active Learning for Multimodal Classification

Meng Shen, Yizheng Huang, Jianxiong Yin, Heqing Zou, Deepu Rajan, Simon See

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.MM

Comments 12 pages, accepted by ACMMM 2023

2308.08558 2023-08-21 q-fin.ST cs.AI 79%

BIRP: Bitcoin Information Retrieval Prediction Model Based on Multimodal Pattern Matching

Minsuk Kim, Byungchul Kim, Junyeong Yong, Jeongwoo Park, Gyeongmin Kim

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments 5 pages, 2 figures, KDD 2023 Machine Learning in Finance workshop

2308.05948 2023-08-14 cs.CV 79%

Uncertainty-Aware Cross-Modal Transfer Network for Sketch-Based 3D Shape Retrieval

Yiyang Cai, Jiaming Lu, Jiewen Wang, Shuang Liang

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 6 pages, 7 figures; To be published in IEEE International Conference on Multimedia and Expo 2023

2302.06350 2023-07-31 cs.CV 79%

VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval

Yan Gong, Georgina Cosma, Axel Finke

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2307.14244 2023-07-27 cs.MM 79%

Neural-based Cross-modal Search and Retrieval of Artwork

Yan Gong, Georgina Cosma, Axel Finke

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.MM

2211.16761 2023-07-25 cs.CV 79%

Improving Cross-Modal Retrieval with Set of Diverse Embeddings

Dongwon Kim, Namyup Kim, Suha Kwak

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2023 (Highlight)

2307.10782 2023-07-21 cs.CV 79%

See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data

Yuhang Lu, Qi Jiang, Runnan Chen, Yuenan Hou, Xinge Zhu, Yuexin Ma

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by ICCV 2023

1411.7798 2023-07-19 cs.CV 79%

Cross-Modal Learning via Pairwise Constraints

Ran He, Man Zhang, Liang Wang, Ye Ji, Qiyue Yin

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 12 pages, 5 figures, 70 references

2306.15796 2023-06-29 cs.AI 79%

ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis

Yakun Yu, Mingjun Zhao, Shi-ang Qi, Feiran Sun, Baoxun Wang, Weidong Guo, Xiaoli Wang, Lei Yang, Di Niu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments Accepted by ACL Findings 2023

2306.06066 2023-06-12 cs.CV cs.LG 79%

Multi-level Cross-modal Feature Alignment via Contrastive Learning towards Zero-shot Classification of Remote Sensing Image Scenes

Chun Liu, Suqiang Ma, Zheng Li, Wei Yang, Zhigang Han

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2306.04272 2023-06-08 cs.CV 79%

On the Generalization of Multi-modal Contrastive Learning

Qi Zhang, Yifei Wang, Yisen Wang

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

2305.17663 2023-05-30 cs.CL 79%

Lexical Retrieval Hypothesis in Multimodal Context

Po-Ya Angela Wang, Pin-Er Chen, Hsin-Yu Chou, Yu-Hsiang Tseng, Shu-Kai Hsieh

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

2305.16566 2023-05-29 cs.CV 79%

Integrating Listwise Ranking into Pairwise-based Image-Text Retrieval

Zheng Li, Caili Guo, Xin Wang, Zerun Feng, Yanjun Wang

专题命中跨模态检索：image-text(title,abstract)；分类 cs.CV

2305.04239 2023-05-09 cs.CV cs.IR 79%

Instance-Variant Loss with Gaussian RBF Kernel for 3D Cross-modal Retriveal

Zhitao Liu, Zengyu Liu, Jiwei Wei, Guan Wang, Zhenjiang Du, Ning Xie, Heng Tao Shen

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2305.01915 2023-05-04 cs.IR cs.MM 79%

Denoising Multi-modal Sequential Recommenders with Contrastive Learning

Dong Yao, Shengyu Zhang, Zhou Zhao, Jieming Zhu, Wenqiao Zhang, Rui Zhang, Xiaofei He, Fei Wu

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.MM

2304.13357 2023-04-27 cs.CV cs.IR 79%

Deep Lifelong Cross-modal Hashing

Liming Xu, Hanqi Li, Bochuan Zheng, Weisheng Li, Jiancheng Lv

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2304.13103 2023-04-27 cs.CR cs.AI 79%

HyMo: Vulnerability Detection in Smart Contracts using a Novel Multi-Modal Hybrid Model

Mohammad Khodadadi, Jafar Tahmoresnezhad

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.AI

2301.04224 2023-04-11 cs.CV cs.LG 79%

Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images

Xindi Wu, KwunFung Lau, Francesco Ferroni, Aljoša Ošep, Deva Ramanan

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 12 pages, 8 figures

2303.14080 2023-03-31 cs.CV 79%

Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data

Paul Hager, Martin J. Menten, Daniel Rueckert

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments Accepted in CVPR 2023

2211.04188 2023-03-28 cs.CV 79%

DepthFormer: Multimodal Positional Encodings and Cross-Input Attention for Transformer-Based Segmentation Networks

Francesco Barbato, Giulia Rizzoli, Pietro Zanuttigh

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at ICASSP 2023

2303.10839 2023-03-22 cs.CV 79%

MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal Representations

Ye Wang, Bowei Jiang, Changqing Zou, Rui Ma

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 16 pages, 14 figures

2303.10249 2023-03-21 eess.IV cs.CV 79%

MRIS: A Multi-modal Retrieval Approach for Image Synthesis on Diverse Modalities

Boqi Chen, Marc Niethammer

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

2201.03597 2023-03-21 cs.CV 79%

Cross-Modality Sub-Image Retrieval using Contrastive Multimodal Image Representations

Eva Breznik, Elisabeth Wetzer, Joakim Lindblad, Nataša Sladoje

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2303.05692 2023-03-13 cs.CV 79%

Semantic-Preserving Augmentation for Robust Image-Text Retrieval

Sunwoo Kim, Kyuhong Shim, Luong Trung Nguyen, Byonghyo Shim

专题命中跨模态检索：image-text(title,abstract)；分类 cs.CV

Comments Accepted to ICASSP 2023

2302.11352 2023-02-23 cs.CV 79%

X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation

Tom van Sonsbeek, Marcel Worring

专题命中跨模态检索：cross-modal(title)；multi-modal(abstract)；分类 cs.CV

Comments IPMI 2023

2210.04754 2023-02-15 cs.CV cs.IR 79%

Improving Visual-Semantic Embeddings by Learning Semantically-Enhanced Hard Negatives for Cross-modal Information Retrieval

Yan Gong, Georgina Cosma

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Journal ref Pattern Recognition 137 (2023): 109272

1. 跨模态检索 3475 篇

Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction

Multimodal Contrastive Learning with Hard Negative Sampling for Human Activity Recognition

Learning with Multi-modal Gradient Attention for Explainable Composed Image Retrieval

MS-Net: A Multi-modal Self-supervised Network for Fine-Grained Classification of Aircraft in SAR Images

Towards Balanced Active Learning for Multimodal Classification

BIRP: Bitcoin Information Retrieval Prediction Model Based on Multimodal Pattern Matching

Uncertainty-Aware Cross-Modal Transfer Network for Sketch-Based 3D Shape Retrieval

VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval

Neural-based Cross-modal Search and Retrieval of Artwork

Improving Cross-Modal Retrieval with Set of Diverse Embeddings

See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data

Cross-Modal Learning via Pairwise Constraints

ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis

Multi-level Cross-modal Feature Alignment via Contrastive Learning towards Zero-shot Classification of Remote Sensing Image Scenes

On the Generalization of Multi-modal Contrastive Learning

Lexical Retrieval Hypothesis in Multimodal Context

Integrating Listwise Ranking into Pairwise-based Image-Text Retrieval

Instance-Variant Loss with Gaussian RBF Kernel for 3D Cross-modal Retriveal

Denoising Multi-modal Sequential Recommenders with Contrastive Learning

Deep Lifelong Cross-modal Hashing

HyMo: Vulnerability Detection in Smart Contracts using a Novel Multi-Modal Hybrid Model

Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images

Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data

DepthFormer: Multimodal Positional Encodings and Cross-Input Attention for Transformer-Based Segmentation Networks

MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal Representations

MRIS: A Multi-modal Retrieval Approach for Image Synthesis on Diverse Modalities

Cross-Modality Sub-Image Retrieval using Contrastive Multimodal Image Representations

Semantic-Preserving Augmentation for Robust Image-Text Retrieval

X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation

Improving Visual-Semantic Embeddings by Learning Semantically-Enhanced Hard Negatives for Cross-modal Information Retrieval