多模态大模型 - arXivDaily 专题

2503.03747 2025-03-06 cs.CR cs.LG 78%

PacketCLIP: Multi-Modal Embedding of Network Traffic and Language for Cybersecurity Reasoning

Ryozo Masukawa, Sanggeon Yun, Sungheon Jeong, Wenjun Huang, Yang Ni, Ian Bryant, Nathaniel D. Bastian, Mohsen Imani

专题命中跨模态检索：multi-modal(title,abstract)

Comments 7 pages, 7 figures

2408.13919 2025-03-05 quant-ph q-bio.NC 78%

Quantum Multimodal Contrastive Learning Framework

Chi-Sheng Chen, Aidan Hung-Wen Tsai, Sheng-Chieh Huang

专题命中跨模态检索：multimodal(title,abstract)

Comments 15 pages

2404.19228 2025-03-04 cs.LG 78%

Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metric

Toshimitsu Uesaka, Taiji Suzuki, Yuhta Takida, Chieh-Hsin Lai, Naoki Murata, Yuki Mitsufuji

专题命中跨模态检索：multimodal(title,abstract)

Comments ICLR 2025 (Spotlight)

2502.15834 2025-02-25 cs.LG stat.ML 78%

Challenges of Multi-Modal Coreset Selection for Depth Prediction

Viktor Moskvoretskii, Narek Alvandian

专题命中跨模态检索：multi-modal(title)；multimodal(abstract)

2311.10567 2025-02-19 cs.IR 78%

Cross-Modal Search and Exploration of Greek Painted Pottery

Elisabeth Trinkl, Stephan Karl, Stefan Lengauer, Reinhold Preiner, Tobias Schreck

专题命中跨模态检索：cross-modal(title)；multimodal(abstract)

Comments 14 pages, 10 figures, preprint for a book chapter, supplementary video available at https://youtu.be/x_Xg0vy3nJY

Journal ref The 3 Dimensions of Digitalised Archaeology: State-of-the-Art, Data Management and Current Challenges in Archaeological 3D-Documentation, Springer International Publishing, 2024

2501.07365 2025-02-18 cs.IR cs.LG 78%

Multimodal semantic retrieval for product search

Dong Liu, Esther Lopez Ramos

专题命中跨模态检索：multimodal(title,abstract)

Comments Accepted at EReL@MIR WWW 2025

2408.16237 2025-02-11 cs.DB 78%

MQRLD: A Multimodal Data Retrieval Platform with Query-aware Feature Representation and Learned Index Based on Data Lake

Ming Sheng, Shuliang Wang, Yong Zhang, Kaige Wang, Jingyi Wang, Yi Luo, Rui Hao

专题命中跨模态检索：multimodal(title,abstract)

Comments 34 pages, 28 figures

2412.11729 2024-12-17 cs.IR 78%

STAIR: Manipulating Collaborative and Multimodal Information for E-Commerce Recommendation

Cong Xu, Yunhang He, Jun Wang, Wei Zhang

专题命中跨模态检索：multimodal(title,abstract)

Comments Accepted at AAAI 2025

2406.08521 2024-11-22 q-bio.CB cs.LG 78%

Embedding-based Multimodal Learning on Pan-Squamous Cell Carcinomas for Improved Survival Outcomes

Asim Waqas, Aakash Tripathi, Paul Stewart, Mia Naeini, Matthew B. Schabath, Ghulam Rasool

专题命中跨模态检索：multimodal(title,abstract)

2411.02837 2024-11-06 cs.LG 78%

On the Comparison between Multi-modal and Single-modal Contrastive Learning

Wei Huang, Andi Han, Yongqiang Chen, Yuan Cao, Zhiqiang Xu, Taiji Suzuki

专题命中跨模态检索：multi-modal(title,abstract)

Comments 51pages, 1 figure, 1 table

Journal ref NeurIPS 2024

2410.23715 2024-11-01 cs.IR 78%

Towards Cross-Modal Text-Molecule Retrieval with Better Modality Alignment

Jia Song, Wanru Zhuang, Yujie Lin, Liang Zhang, Chunyan Li, Jinsong Su, Song He, Xiaochen Bo

专题命中跨模态检索：cross-modal(title,abstract)

Comments BIBM 2024 regular paper

2410.15624 2024-10-22 cs.LG 78%

Test-time Adaptation for Cross-modal Retrieval with Query Shift

Haobin Li, Peng Hu, Qianjun Zhang, Xi Peng, Xiting Liu, Mouxing Yang

专题命中跨模态检索：cross-modal(title,abstract)

Comments 22 pages, 8 figures

2410.15387 2024-10-22 cs.IR 78%

Deep Class-guided Hashing for Multi-label Cross-modal Retrieval

Hao Chen, Lei Zhu, Xinghui Zhu

专题命中跨模态检索：cross-modal(title,abstract)

2410.11376 2024-10-16 cs.CE 78%

PhysioFormer: Integrating Multimodal Physiological Signals and Symbolic Regression for Explainable Affective State Prediction

Zhifeng Wang, Wanxuan Wu, Chunyan Zeng

专题命中跨模态检索：multimodal(title,abstract)

Comments 45 pages

2409.02965 2024-09-06 cs.SI cs.IR cs.LG 78%

Do We Trust What They Say or What They Do? A Multimodal User Embedding Provides Personalized Explanations

Zhicheng Ren, Zhiping Xiao, Yizhou Sun

专题命中跨模态检索：multimodal(title,abstract)

2101.08316 2024-08-27 cs.LG 78%

Ensemble manifold based regularized multi-modal graph convolutional network for cognitive ability prediction

Gang Qu, Li Xiao, Wenxing Hu, Kun Zhang, Vince D. Calhoun, Yu-Ping Wang

专题命中跨模态检索：multi-modal(title,abstract)

2408.07773 2024-08-16 cs.LG 78%

MedTsLLM: Leveraging LLMs for Multimodal Medical Time Series Analysis

Nimeesha Chan, Felix Parker, William Bennett, Tianyi Wu, Mung Yao Jia, James Fackler, Kimia Ghobadi

专题命中跨模态检索：multimodal(title,abstract)

Comments published in Proceedings of Machine Learning Research, MLHC 2024

2408.00665 2024-08-02 cs.LG 78%

AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Models

Daqin Luo, Chengjian Feng, Yuxuan Nong, Yiqing Shen

专题命中跨模态检索：multimodal(title,abstract)

Comments Accpeted by ACMMM2024

2407.11696 2024-07-17 cs.LG physics.ao-ph 78%

Global atmospheric data assimilation with multi-modal masked autoencoders

Thomas J. Vandal, Kate Duffy, Daniel McDuff, Yoni Nachmany, Chris Hartshorn

专题命中跨模态检索：multi-modal(title,abstract)

Comments 24 pages, 9 figures, 6 tables

2407.04217 2024-07-08 cs.DB cs.IR 78%

An Interactive Multi-modal Query Answering System with Retrieval-Augmented Large Language Models

Mengzhao Wang, Haotian Wu, Xiangyu Ke, Yunjun Gao, Xiaoliang Xu, Lu Chen

专题命中跨模态检索：multi-modal(title,abstract)

Comments This demo paper has been accepted by VLDB 2024

2407.02238 2024-07-03 cs.DC cs.LG cs.PF 78%

MIREncoder: Multi-modal IR-based Pretrained Embeddings for Performance Optimizations

Akash Dutta, Ali Jannesari

专题命中跨模态检索：multi-modal(title,abstract)

Comments 12 pages, 6 figures, 9 tables, PACT '24 conference

2302.05342 2024-06-27 cs.LG 78%

Combining Reconstruction and Contrastive Methods for Multimodal Representations in RL

Philipp Becker, Sebastian Mossburger, Fabian Otto, Gerhard Neumann

专题命中跨模态检索：multimodal(title,abstract)

Comments Published in "Reinforcement Learning Conference (RLC)", August 2024

2310.14037 2024-06-18 cs.IR 78%

MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin

Tianshuo Zhou, Sen Mei, Xinze Li, Zhenghao Liu, Chenyan Xiong, Zhiyuan Liu, Yu Gu, Ge Yu

专题命中跨模态检索：multi-modal(title,abstract)

2305.03972 2024-06-11 cs.IR 78%

Category-Oriented Representation Learning for Image to Multi-Modal Retrieval

Zida Cheng, Chen Ju, Shuai Xiao, Xu Chen, Zhonghua Zhai, Xiaoyi Zeng, Weilin Huang, Junchi Yan

专题命中跨模态检索：multi-modal(title,abstract)

2405.20668 2024-06-03 q-bio.BM cs.LG q-bio.QM 78%

Improving Paratope and Epitope Prediction by Multi-Modal Contrastive Learning and Interaction Informativeness Estimation

Zhiwei Wang, Yongkang Wang, Wen Zhang

专题命中跨模态检索：multi-modal(title,abstract)

Comments This paper is accepted by IJCAI 2024

2311.05956 2024-05-24 cs.IR cs.LG 78%

ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation

Yuting Liu, Enneng Yang, Yizhou Dang, Guibing Guo, Qiang Liu, Yuliang Liang, Linying Jiang, Xingwei Wang

专题命中跨模态检索：multimodal(title,abstract)

2312.16400 2024-04-19 cs.IR 78%

LGMRec: Local and Global Graph Learning for Multimodal Recommendation

Zhiqiang Guo, Jianjun Li, Guohui Li, Chaoyang Wang, Si Shi, Bin Ruan

专题命中跨模态检索：multimodal(title,abstract)

Comments Accepted by AAAI2024

2404.06078 2024-04-10 cs.IR 78%

End-to-end training of Multimodal Model and ranking Model

Xiuqi Deng, Lu Xu, Xiyao Li, Jinkai Yu, Erpeng Xue, Zhongyuan Wang, Di Zhang, Zhaojie Liu, Guorui Zhou, Yang Song, Na Mou, Shen Jiang, Han Li

专题命中跨模态检索：multimodal(title,abstract)

Comments 9 pages, 8 figures

2403.18316 2024-03-28 cs.LG 78%

Multi-Modal Contrastive Learning for Online Clinical Time-Series Applications

Fabian Baldenweg, Manuel Burger, Gunnar Rätsch, Rita Kuznetsova

专题命中跨模态检索：multi-modal(title,abstract)

Comments Accepted as a Workshop Paper at TS4H@ICLR2024

2403.09428 2024-03-27 cs.LG 78%

Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity

Zhuo Zhi, Ziquan Liu, Moe Elbadawi, Adam Daneshmend, Mine Orlu, Abdul Basit, Andreas Demosthenous, Miguel Rodrigues

专题命中跨模态检索：multimodal(title,abstract)

1. 跨模态检索 3475 篇

PacketCLIP: Multi-Modal Embedding of Network Traffic and Language for Cybersecurity Reasoning

Quantum Multimodal Contrastive Learning Framework

Weighted Point Set Embedding for Multimodal Contrastive Learning Toward Optimal Similarity Metric

Challenges of Multi-Modal Coreset Selection for Depth Prediction

Cross-Modal Search and Exploration of Greek Painted Pottery

Multimodal semantic retrieval for product search

MQRLD: A Multimodal Data Retrieval Platform with Query-aware Feature Representation and Learned Index Based on Data Lake

STAIR: Manipulating Collaborative and Multimodal Information for E-Commerce Recommendation

Embedding-based Multimodal Learning on Pan-Squamous Cell Carcinomas for Improved Survival Outcomes

On the Comparison between Multi-modal and Single-modal Contrastive Learning

Towards Cross-Modal Text-Molecule Retrieval with Better Modality Alignment

Test-time Adaptation for Cross-modal Retrieval with Query Shift

Deep Class-guided Hashing for Multi-label Cross-modal Retrieval

PhysioFormer: Integrating Multimodal Physiological Signals and Symbolic Regression for Explainable Affective State Prediction

Do We Trust What They Say or What They Do? A Multimodal User Embedding Provides Personalized Explanations

Ensemble manifold based regularized multi-modal graph convolutional network for cognitive ability prediction

MedTsLLM: Leveraging LLMs for Multimodal Medical Time Series Analysis

AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Models

Global atmospheric data assimilation with multi-modal masked autoencoders

An Interactive Multi-modal Query Answering System with Retrieval-Augmented Large Language Models

MIREncoder: Multi-modal IR-based Pretrained Embeddings for Performance Optimizations

Combining Reconstruction and Contrastive Methods for Multimodal Representations in RL

MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin

Category-Oriented Representation Learning for Image to Multi-Modal Retrieval

Improving Paratope and Epitope Prediction by Multi-Modal Contrastive Learning and Interaction Informativeness Estimation

ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation

LGMRec: Local and Global Graph Learning for Multimodal Recommendation

End-to-end training of Multimodal Model and ranking Model

Multi-Modal Contrastive Learning for Online Clinical Time-Series Applications

Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity