多模态大模型

Text-Region Matching for Multi-Label Image Recognition with Missing Labels

Leilei Ma, Hongxing Xie, Lei Wang, Yanping Fu, Dengdi Sun, Haifeng Zhao

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted to ACM International Conference on Multimedia (ACM MM) 2024

ALS-HAR: Harnessing Wearable Ambient Light Sensors to Enhance IMU-based Human Activity Recogntion

Lala Shakti Swarup Ray, Daniel Geißler, Mengxi Liu, Bo Zhou, Sungho Suh, Paul Lukowicz

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.AI

Masks and Manuscripts: Advancing Medical Pre-training with End-to-End Masking and Narrative Structuring

Shreyank N Gowda, David A. Clifton

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted in MICCAI-24

Text-Video Retrieval with Global-Local Semantic Consistent Learning

Haonan Zhang, Pengpeng Zeng, Lianli Gao, Jingkuan Song, Yihang Duan, Xinyu Lyu, Hengtao Shen

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments The author has withdrawn this paper due to a critical definitional error in concept learning for global/local-interaction learning during training. This error led to an alignment issue with the definition of the text-video retrieval task, causing an unfair comparison with state-of-the-art (SOTA) methods. Consequently, this hindered the accurate evaluation of the paper's contributions

Second Place Solution of WSDM2023 Toloka Visual Question Answering Challenge

Xiangyu Wu, Zhouyang Chi, Yang Yang, Jianfeng Lu

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Second Place of WSDM2023 Toloka Visual Question Answering Challenge

Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning

Shuvendu Roy, Yasaman Parhizkar, Franklin Ogidi, Vahid Reza Khazaie, Michael Colacci, Ali Etemad, Elham Dolatabadi, Arash Afkanpour

专题命中 跨模态检索 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Confidence-aware multi-modality learning for eye disease screening

Ke Zou, Tian Lin, Zongbo Han, Meng Wang, Xuedong Yuan, Haoyu Chen, Changqing Zhang, Xiaojing Shen, Huazhu Fu

专题命中 跨模态检索 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments 27 pages, 7 figures, 9 tables

H2ASeg: Hierarchical Adaptive Interaction and Weighting Network for Tumor Segmentation in PET/CT Images

Jinpeng Lu, Jingyun Chen, Linghan Cai, Songhan Jiang, Yongbing Zhang

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 10 pages,4 figures

SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training

Kazem Meidani, Parshin Shojaee, Chandan K. Reddy, Amir Barati Farimani

专题命中 跨模态检索 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.AI

Comments ICLR 2024 Spotlight Paper

Zero-shot Composed Text-Image Retrieval

Yikun Liu, Jiangchao Yao, Ya Zhang, Yanfeng Wang, Weidi Xie

专题命中 跨模态检索 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Retrieval-Enhanced Contrastive Vision-Text Models

Ahmet Iscen, Mathilde Caron, Alireza Fathi, Cordelia Schmid

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

COLA: A Benchmark for Compositional Text-to-image Retrieval

Arijit Ray, Filip Radenovic, Abhimanyu Dubey, Bryan A. Plummer, Ranjay Krishna, Kate Saenko

专题命中 跨模态检索 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2023. Webpage: https://cs-people.bu.edu/array/research/cola/

CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders

Anthony Fuller, Koreen Millard, James R. Green

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments NeurIPS 2023 Camera Ready

Factorized Contrastive Learning: Going Beyond Multi-view Redundancy

Paul Pu Liang, Zihao Deng, Martin Ma, James Zou, Louis-Philippe Morency, Ruslan Salakhutdinov

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments NeurIPS 2023. Code available at: https://github.com/pliang279/FactorCL

PaLI-3 Vision Language Models: Smaller, Faster, Stronger

Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov, Jialin Wu, Paul Voigtlaender, Basil Mustafa, Sebastian Goodman, Ibrahim Alabdulmohsin, Piotr Padlewski, Daniel Salz, Xi Xiong, Daniel Vlasic, Filip Pavetic, Keran Rong, Tianli Yu, Daniel Keysers, Xiaohua Zhai, Radu Soricut

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

NEUCORE: Neural Concept Reasoning for Composed Image Retrieval

Shu Zhao, Huijuan Xu

专题命中 跨模态检索 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Self-Supervised Contrastive Learning for Robust Audio-Sheet Music Retrieval Systems

Luis Carvalho, Tobias Washüttl, Gerhard Widmer

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 eess.AS

Journal ref Proceedings of the 14th ACM Multimedia Systems Conference (MMSys '23), June 7-10, 2023, Vancouver, BC, Canada

Improving Face Recognition from Caption Supervision with Multi-Granular Contextual Feature Aggregation

Md Mahedi Hasan, Nasser Nasrabadi

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments This article has been accepted for publication in the IEEE International Joint Conference on Biometrics (IJCB), 2023

UGIF: UI Grounded Instruction Following

Sagar Gubbi Venkatesh, Partha Talukdar, Srini Narayanan

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CL

IMAGINATOR: Pre-Trained Image+Text Joint Embeddings using Word-Level Grounding of Images

Varuna Krishna, S Suryavardan, Shreyash Mishra, Sathyanarayanan Ramamoorthy, Parth Patwa, Megha Chakraborty, Aman Chadha, Amitava Das, Amit Sheth

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Learning Robust Visual-Semantic Embedding for Generalizable Person Re-identification

Suncheng Xiang, Jingsheng Gao, Mengyuan Guan, Jiacheng Ruan, Chengfeng Zhou, Ting Liu, Dahong Qian, Yuzhuo Fu

专题命中 跨模态检索 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

CLIPPO: Image-and-Language Understanding from Pixels Only

Michael Tschannen, Basil Mustafa, Neil Houlsby

专题命中 跨模态检索 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2023. Code and pretrained models are available at https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/clippo/README.md

Large Scale Visual Food Recognition

Weiqing Min, Zhiling Wang, Yuxin Liu, Mengjiang Luo, Liping Kang, Xiaoming Wei, Xiaolin Wei, Shuqiang Jiang

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence

Bridge the Gap between Language models and Tabular Understanding

Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Chenyu You, Jianhui Chang, Daxin Jiang, Jia Li

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CL

Comments 7 pages

Normalized Contrastive Learning for Text-Video Retrieval

Yookoon Park, Mahmoud Azab, Bo Xiong, Seungwhan Moon, Florian Metze, Gourab Kundu, Kirmani Ahmed

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Published in EMNLP 2022

CyCLIP: Cyclic Contrastive Language-Image Pretraining

Shashank Goel, Hritik Bansal, Sumit Bhatia, Ryan A. Rossi, Vishwa Vinay, Aditya Grover

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 19 pages, 13 tables, 6 figures, Oral at NeuRIPS 2022

Weakly Supervised Face Naming with Symmetry-Enhanced Contrastive Loss

Tingyu Qu, Tinne Tuytelaars, Marie-Francine Moens

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023

Adaptive Asymmetric Label-guided Hashing for Multimedia Search

Yitian Long

专题命中 跨模态检索 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.MM

Comments Submitted to APPLIED SCIENCES

Probing Visual-Audio Representation for Video Highlight Detection via Hard-Pairs Guided Contrastive Learning

Shuaicheng Li, Feng Zhang, Kunlin Yang, Lingbo Liu, Shinan Liu, Jun Hou, Shuai Yi

专题命中 跨模态检索 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Progressive Learning for Image Retrieval with Hybrid-Modality Queries

Yida Zhao, Yuqing Song, Qin Jin

专题命中 跨模态检索 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by SIGIR 2022