多模态大模型 - arXivDaily 专题

2401.08212 2024-04-16 cs.CV 57%

Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication

Hanjia Lyu, Weihong Qi, Zhongyu Wei, Jiebo Luo

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CV

Comments Accepted for publication in ICWSM 2024

2310.06671 2024-04-16 cs.CL 57%

Making Large Language Models Perform Better in Knowledge Graph Completion

Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Wen Zhang, Huajun Chen

专题命中多模态训练与对齐：cross-modal(abstract)；分类 cs.CL

Comments Working in progress

2404.08406 2024-04-15 cs.CV 57%

MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion

Zhe Li, Haiwei Pan, Kejia Zhang, Yuhua Wang, Fengming Yu

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

2404.07790 2024-04-12 cs.CV 57%

VIFNet: An End-to-end Visible-Infrared Fusion Network for Image Dehazing

Meng Yu, Te Cui, Haoyang Lu, Yufeng Yue

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CV

2401.16923 2024-04-12 cs.CV cs.RO eess.IV 57%

Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation

Ruiping Liu, Jiaming Zhang, Kunyu Peng, Yufan Chen, Ke Cao, Junwei Zheng, M. Saquib Sarfraz, Kailun Yang, Rainer Stiefelhagen

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments Accepted to IEEE IV 2024. The source code is publicly available at https://github.com/RuipingL/MISS

2404.06962 2024-04-11 cs.LG cs.AI 57%

Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study

Hongru Du, Jianan Zhao, Yang Zhao, Shaochong Xu, Xihong Lin, Yiran Chen, Lauren M. Gardner, Hao Frank Yang

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.AI

Comments 35 pages, 10 figures

2404.06911 2024-04-11 cs.CL 57%

GraSAME: Injecting Token-Level Structural Information to Pretrained Language Models via Graph-guided Self-Attention Mechanism

Shuzhou Yuan, Michael Färber

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CL

Comments NAACL 2024 Findings

2311.18649 2024-04-10 cs.CV 57%

Simple Semantic-Aided Few-Shot Learning

Hai Zhang, Junzhe Xu, Shanlin Jiang, Zhenan He

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2404.05183 2024-04-09 cs.CV cs.LG 57%

Progressive Alignment with VLM-LLM Feature to Augment Defect Classification for the ASE Dataset

Chih-Chung Hsu, Chia-Ming Lee, Chun-Hung Sun, Kuang-Ming Wu

专题命中多模态训练与对齐：image-text(abstract)；分类 cs.CV

Comments MULA 2024

2404.04971 2024-04-09 cs.CV 57%

FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation

Jianghao Wu, Dong Guo, Guotai Wang, Qiang Yue, Huijun Yu, Kang Li, Shaoting Zhang

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

2403.16834 2024-04-09 cs.CV 57%

From Two-Stream to One-Stream: Efficient RGB-T Tracking via Mutual Prompt Learning and Knowledge Distillation

Yang Luo, Xiqing Guo, Hao Li

专题命中多模态训练与对齐：cross-modal(abstract)；分类 cs.CV

2312.01987 2024-04-05 cs.CV 57%

Bootstrapping SparseFormers from Vision Foundation Models

Ziteng Gao, Zhan Tong, Kevin Qinghong Lin, Joya Chen, Mike Zheng Shou

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CV

Comments CVPR 2024

2311.16926 2024-04-04 cs.CV 57%

LLaFS: When Large Language Models Meet Few-Shot Segmentation

Lanyun Zhu, Tianrun Chen, Deyi Ji, Jieping Ye, Jun Liu

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments Accepted to CVPR2024

2404.01074 2024-04-02 cs.CV cs.LG 57%

Prompt Learning for Oriented Power Transmission Tower Detection in High-Resolution SAR Images

Tianyang Li, Chao Wang, Hong Zhang

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CV

Comments 22 pages, 12figures

2402.18933 2024-04-02 cs.CV 57%

Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration

Tony C. W. Mok, Zi Li, Yunhao Bai, Jianpeng Zhang, Wei Liu, Yan-Jie Zhou, Ke Yan, Dakai Jin, Yu Shi, Xiaoli Yin, Le Lu, Ling Zhang

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CV

Comments Accepted by CVPR2024

2209.02200 2024-03-28 cs.CV 57%

Task-wise Sampling Convolutions for Arbitrary-Oriented Object Detection in Aerial Images

Zhanchao Huang, Wei Li, Xiang-Gen Xia, Hao Wang, Ran Tao

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CV

Comments 15 pages, 13 figures, 11 tables

Journal ref IEEE Transactions on Neural Networks and Learning Systems,2024

2403.16440 2024-03-26 cs.CV 57%

RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection

Zhiwei Lin, Zhe Liu, Zhongyu Xia, Xinhao Wang, Yongtao Wang, Shengxiang Qi, Yang Dong, Nan Dong, Le Zhang, Ce Zhu

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR2024

2403.16387 2024-03-26 cs.CV 57%

Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion

Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang, Jiayi Ma

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2403.15673 2024-03-26 cs.CL 57%

AI for Biomedicine in the Era of Large Language Models

Zhenyu Bi, Sajib Acharjee Dip, Daniel Hajialigol, Sindhura Kommu, Hanwen Liu, Meng Lu, Xuan Wang

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CL

Comments 8 pages, 3 figures

2403.12494 2024-03-26 cs.CV 57%

Task-Customized Mixture of Adapters for General Image Fusion

Pengfei Zhu, Yang Sun, Bing Cao, Qinghua Hu

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2402.17417 2024-03-26 cs.CV 57%

CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification

Haoran Lai, Qingsong Yao, Zihang Jiang, Rongsheng Wang, Zhiyang He, Xiaodong Tao, S. Kevin Zhou

专题命中多模态训练与对齐：image-text(abstract)；分类 cs.CV

2312.12198 2024-03-26 cs.CV 57%

Mask Grounding for Referring Image Segmentation

Yong Xien Chng, Henry Zheng, Yizeng Han, Xuchong Qiu, Gao Huang

专题命中多模态训练与对齐：cross-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR2024; Project page: https://yxchng.github.io/projects/mask-grounding

2305.16582 2024-03-26 cs.CL 57%

Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models

Yao Yao, Zuchao Li, Hai Zhao

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CL

2403.15257 2024-03-25 cs.SI cs.AI 57%

Hierarchical Information Enhancement Network for Cascade Prediction in Social Networks

Fanrui Zhang, Jiawei Liu, Qiang Zhang, Xiaoling Zhu, Zheng-Jun Zha

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.AI

Comments 7 pages, 2 figures

2309.04669 2024-03-25 cs.CV 57%

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Yang Jin, Kun Xu, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Quzhe Huang, Bin Chen, Chenyi Lei, An Liu, Chengru Song, Xiaoqiang Lei, Di Zhang, Wenwu Ou, Kun Gai, Yadong Mu

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments ICLR 2024

2211.06687 2024-03-25 cs.SD eess.AS 57%

Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation

Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Marianna Nezhurina, Taylor Berg-Kirkpatrick, Shlomo Dubnov

专题命中多模态训练与对齐：multimodal(abstract)；分类 eess.AS

2403.12532 2024-03-20 cs.CV 57%

UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All

Yuanhuiyi Lyu, Xu Zheng, Jiazhou Zhou, Lin Wang

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

Comments Accepted to CVPR2024

2203.07738 2024-03-20 cs.CV 57%

GCT: Graph Co-Training for Semi-Supervised Few-Shot Learning

Rui Xu, Lei Xing, Shuai Shao, Lifei Zhao, Baodi Liu, Weifeng Liu, Yicong Zhou

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

2403.11373 2024-03-19 cs.CV 57%

Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration

Shu Zhao, Xiaohan Zou, Tan Yu, Huijuan Xu

专题命中多模态训练与对齐：multi-modal(abstract)；分类 cs.CV

2308.06668 2024-03-19 cs.LG cs.CV 57%

Large Language Models and Foundation Models in Smart Agriculture: Basics, Opportunities, and Challenges

Jiajia Li, Mingle Xu, Lirong Xiang, Dong Chen, Weichao Zhuang, Xunyuan Yin, Zhaojian Li

专题命中多模态训练与对齐：multimodal(abstract)；分类 cs.CV

Comments 18 pages, 3 figures

1. 多模态训练与对齐 6929 篇

Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication

Making Large Language Models Perform Better in Knowledge Graph Completion

MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion

VIFNet: An End-to-end Visible-Infrared Fusion Network for Image Dehazing

Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation

Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study

GraSAME: Injecting Token-Level Structural Information to Pretrained Language Models via Graph-guided Self-Attention Mechanism

Simple Semantic-Aided Few-Shot Learning

Progressive Alignment with VLM-LLM Feature to Augment Defect Classification for the ASE Dataset

FPL+: Filtered Pseudo Label-based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation

From Two-Stream to One-Stream: Efficient RGB-T Tracking via Mutual Prompt Learning and Knowledge Distillation

Bootstrapping SparseFormers from Vision Foundation Models

LLaFS: When Large Language Models Meet Few-Shot Segmentation

Prompt Learning for Oriented Power Transmission Tower Detection in High-Resolution SAR Images

Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration

Task-wise Sampling Convolutions for Arbitrary-Oriented Object Detection in Aerial Images

RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detection

Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion

AI for Biomedicine in the Era of Large Language Models

Task-Customized Mixture of Adapters for General Image Fusion

CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification

Mask Grounding for Referring Image Segmentation

Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models

Hierarchical Information Enhancement Network for Cascade Prediction in Social Networks

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation

UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All

GCT: Graph Co-Training for Semi-Supervised Few-Shot Learning

Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration

Large Language Models and Foundation Models in Smart Agriculture: Basics, Opportunities, and Challenges