多模态大模型 - arXivDaily 专题

2411.16020 2024-11-26 cs.CL 79%

TransCompressor: LLM-Powered Multimodal Data Compression for Smart Transportation

Huanqi Yang, Rucheng Wu, Weitao Xu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments 6 pages

2411.15576 2024-11-26 eess.IV cs.CV 79%

MulModSeg: Enhancing Unpaired Multi-Modal Medical Image Segmentation with Modality-Conditioned Text Embedding and Alternating Training

Chengyin Li, Hui Zhu, Rafi Ibn Sultan, Hassan Bagher Ebadian, Prashant Khanduri, Chetty Indrin, Kundan Thind, Dongxiao Zhu

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by WACV-2025

2411.14704 2024-11-25 cs.CV cs.IR 79%

Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval

Zengbao Sun, Ming Zhao, Gaorui Liu, André Kaup

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Journal ref IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1-18, 2024, Art no. 4709118

2411.06229 2024-11-12 cs.AI 79%

Multimodal Contrastive Learning of Urban Space Representations from POI Data

Xinglei Wang, Tao Cheng, Stephen Law, Zichao Zeng, Lu Yin, Junyuan Liu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments 19 pages, 5 figures, 7 tables

2411.06176 2024-11-12 cs.CL 79%

M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework

Yew Ken Chia, Liying Cheng, Hou Pong Chan, Chaoqun Liu, Maojia Song, Sharifah Mahani Aljunied, Soujanya Poria, Lidong Bing

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

2411.05900 2024-11-12 cs.CV cs.LG 79%

Enhancing Cardiovascular Disease Prediction through Multi-Modal Self-Supervised Learning

Francesco Girlanda, Olga Demler, Bjoern Menze, Neda Davoudi

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to British Machine Vision Conference (BMVC) 2024

2411.02534 2024-11-06 eess.IV cs.CV 79%

Multi-modal Spatial Clustering for Spatial Transcriptomics Utilizing High-resolution Histology Images

Bingjun Li, Mostafa Karami, Masum Shah Junayed, Sheida Nabavi

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments 9 pages

2410.18104 2024-10-25 cs.NI cs.AI 79%

ENWAR: A RAG-empowered Multi-Modal LLM Framework for Wireless Environment Perception

Ahmad M. Nazar, Abdulkadir Celik, Mohamed Y. Selim, Asmaa Abdallah, Daji Qiao, Ahmed M. Eltawil

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.AI

2410.11321 2024-10-16 cs.CL 79%

Self-adaptive Multimodal Retrieval-Augmented Generation

Wenjia Zhai

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

2410.06285 2024-10-10 cs.CV cs.RO 79%

Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching

Gongxin Yao, Xinyang Li, Luowei Fu, Yu Pan

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2403.13395 2024-10-10 cs.CV cs.RO 79%

Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments

Alberto García-Hernández, Riccardo Giubilato, Klaus H. Strobl, Javier Civera, Rudolph Triebel

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments Accepted submission to International Conference on Robotics and Automation (ICRA), 2024

Journal ref 2024 IEEE International Conference on Robotics and Automation (ICRA)

2410.01160 2024-10-03 cs.IR cs.CV 79%

GraphRevisedIE: Multimodal Information Extraction with Graph-Revised Network

Panfeng Cao, Jian Wu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Journal ref Pattern Recognition Volume 140, August 2023, 109542

2409.17055 2024-10-02 cs.AI cs.LG 79%

DRIM: Learning Disentangled Representations from Incomplete Multimodal Healthcare Data

Lucas Robinet, Ahmad Berjaoui, Ziad Kheil, Elizabeth Cohen-Jonathan Moyal

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

2409.15342 2024-09-25 cs.IR cs.AI cs.LG 79%

Recall: Empowering Multimodal Embedding for Edge Devices

Dongqi Cai, Shangguang Wang, Chen Peng, Zeling Zhang, Mengwei Xu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

2409.11624 2024-09-19 cs.CV cs.LG 79%

Multimodal Generalized Category Discovery

Yuchang Su, Renping Zhou, Siyu Huang, Xingjian Li, Tianyang Wang, Ziyue Wang, Min Xu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2408.16029 2024-09-16 cs.LG cs.AI 79%

Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis

Sijie Mai, Yu Zhao, Ying Zeng, Jianhua Yao, Haifeng Hu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

2409.06450 2024-09-11 cs.RO cs.AI cs.ET 79%

Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

Qiujing Lu, Xuanhan Wang, Yiwei Jiang, Guangming Zhao, Mingyue Ma, Shuo Feng

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

2403.10158 2024-09-11 cs.LG cs.AI 79%

Functional Graph Convolutional Networks: A unified multi-task and multi-modal learning framework to facilitate health and social-care insights

Tobia Boschi, Francesca Bonin, Rodrigo Ordonez-Hurtado, Cécile Rousseau, Alessandra Pascale, John Dinsmore

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.AI

Journal ref IJCAI 2024

2308.09012 2024-09-10 cs.CV 79%

FashionLOGO: Prompting Multimodal Large Language Models for Fashion Logo Embeddings

Zhen Wang, Da Li, Yulin Su, Min Yang, Minghui Qiu, Walton Wang

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2303.11783 2024-09-05 q-bio.BM cs.AI cs.LG 79%

CCPL: Cross-modal Contrastive Protein Learning

Jiangbin Zheng, Stan Z. Li

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.AI

Comments Accepted to ICPR 2024

2408.16296 2024-08-30 cs.CV cs.IR 79%

Rethinking Sparse Lexical Representations for Image Retrieval in the Age of Rising Multi-Modal Large Language Models

Kengo Nakata, Daisuke Miyashita, Youyang Ng, Yasuto Hoshi, Jun Deguchi

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to ECCV 2024 Workshops: 2nd Workshop on Traditional Computer Vision in the Age of Deep Learning (TradiCV)

2311.14902 2024-08-27 cs.CV 79%

Parkinson's Disease Classification Using Contrastive Graph Cross-View Learning with Multimodal Fusion of SPECT Images and Clinical Features

Jun-En Ding, Chien-Chin Hsu, Feng Liu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2401.01524 2024-08-21 cs.CV 79%

Multimodal self-supervised learning for lesion localization

Hao Yang, Hong-Yu Zhou, Cheng Li, Weijian Huang, Jiarun Liu, Yong Liang, Guangming Shi, Hairong Zheng, Qiegen Liu, Shanshan Wang

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2408.03922 2024-08-08 cs.CV 79%

FMiFood: Multi-modal Contrastive Learning for Food Image Classification

Xinyue Pan, Jiangpeng He, Fengqing Zhu

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

2407.15376 2024-07-23 cs.MM 79%

Structure-Aware Residual-Center Representation for Self-Supervised Open-Set 3D Cross-Modal Retrieval

Yang Xu, Yifan Feng, Yu Jiang

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.MM

Comments ICME 2024

2303.11879 2024-07-23 cs.IR cs.MM 79%

Multimodal Pre-training Framework for Sequential Recommendation via Contrastive Learning

Lingzi Zhang, Xin Zhou, Zhiwei Zeng, Zhiqi Shen

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.MM

Comments 22 pages

2407.14230 2024-07-22 cs.CV cs.LG 79%

ETSCL: An Evidence Theory-Based Supervised Contrastive Learning Framework for Multi-modal Glaucoma Grading

Zhiyuan Yang, Bo Zhang, Yufei Shi, Ningze Zhong, Johnathan Loh, Huihui Fang, Yanwu Xu, Si Yong Yeo

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by Ophthalmic Medical Image Analysis Workshop at MICCAI'24

2407.13221 2024-07-19 cs.CV 79%

Multimodal Label Relevance Ranking via Reinforcement Learning

Taian Guo, Taolin Zhang, Haoqian Wu, Hanjun Li, Ruizhi Qiao, Xing Sun

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to ECCV2024

2407.12025 2024-07-18 cs.HC cs.AI 79%

LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

Ran Chen, Xueqi Yao, Xuhui Jiang

专题命中跨模态检索：multi-modal(title)；cross-modal(abstract)；分类 cs.AI

2407.10714 2024-07-16 cs.IR cs.AI 79%

SEMINAR: Search Enhanced Multi-modal Interest Network and Approximate Retrieval for Lifelong Sequential Recommendation

Kaiming Shen, Xichen Ding, Zixiang Zheng, Yuqi Gong, Qianqian Li, Zhongyi Liu, Guannan Zhang

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.AI

Comments 9 pages,code released

1. 跨模态检索 3475 篇

TransCompressor: LLM-Powered Multimodal Data Compression for Smart Transportation

MulModSeg: Enhancing Unpaired Multi-Modal Medical Image Segmentation with Modality-Conditioned Text Embedding and Alternating Training

Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval

Multimodal Contrastive Learning of Urban Space Representations from POI Data

M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework

Enhancing Cardiovascular Disease Prediction through Multi-Modal Self-Supervised Learning

Multi-modal Spatial Clustering for Spatial Transcriptomics Utilizing High-resolution Histology Images

ENWAR: A RAG-empowered Multi-Modal LLM Framework for Wireless Environment Perception

Self-adaptive Multimodal Retrieval-Augmented Generation

Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching

Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments

GraphRevisedIE: Multimodal Information Extraction with Graph-Revised Network

DRIM: Learning Disentangled Representations from Incomplete Multimodal Healthcare Data

Recall: Empowering Multimodal Embedding for Edge Devices

Multimodal Generalized Category Discovery

Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis

Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

Functional Graph Convolutional Networks: A unified multi-task and multi-modal learning framework to facilitate health and social-care insights

FashionLOGO: Prompting Multimodal Large Language Models for Fashion Logo Embeddings

CCPL: Cross-modal Contrastive Protein Learning

Rethinking Sparse Lexical Representations for Image Retrieval in the Age of Rising Multi-Modal Large Language Models

Parkinson's Disease Classification Using Contrastive Graph Cross-View Learning with Multimodal Fusion of SPECT Images and Clinical Features

Multimodal self-supervised learning for lesion localization

FMiFood: Multi-modal Contrastive Learning for Food Image Classification

Structure-Aware Residual-Center Representation for Self-Supervised Open-Set 3D Cross-Modal Retrieval

Multimodal Pre-training Framework for Sequential Recommendation via Contrastive Learning

ETSCL: An Evidence Theory-Based Supervised Contrastive Learning Framework for Multi-modal Glaucoma Grading

Multimodal Label Relevance Ranking via Reinforcement Learning

LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

SEMINAR: Search Enhanced Multi-modal Interest Network and Approximate Retrieval for Lifelong Sequential Recommendation