多模态大模型

RelVAE: Generative Pretraining for few-shot Visual Relationship Detection

Sotiris Karapiperis, Markos Diomataris, Vassilis Pitsikalis

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features

Mingxin Liu, Yunzan Liu, Hui Cui, Chunquan Li, Jiquan Ma

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 4 figures, accepted by 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM 2023)

Dual-Branch Reconstruction Network for Industrial Anomaly Detection with RGB-D Data

Chenyang Bi, Yueyang Li, Haichi Luo

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 5 figures

PerceptionGPT: Effectively Fusing Visual Perception into LLM

Renjie Pi, Lewei Yao, Jiahui Gao, Jipeng Zhang, Tong Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding

Morris Alper, Michael Fiman, Hadar Averbuch-Elor

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR 2023. Project webpage: https://isbertblind.github.io/

Debunking Free Fusion Myth: Online Multi-view Anomaly Detection with Disentangled Product-of-Experts Modeling

Hao Wang, Zhi-Qi Cheng, Jingdong Sun, Xin Yang, Xiao Wu, Hongyang Chen, Yan Yang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ACM Multimedia 2023, 10 pages, 5 tables, and 3 figures

RGB-X Object Detection via Scene-Specific Fusion Modules

Sri Aditya Deevi, Connor Lee, Lu Gan, Sushruth Nagesh, Gaurav Pandey, Soon-Jo Chung

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024)

Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification

Rui Wang, Peipei Li, Huaibo Huang, Chunshui Cao, Ran He, Zhaofeng He

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2023

STERLING: Self-Supervised Terrain Representation Learning from Unconstrained Robot Experience

Haresh Karnan, Elvin Yang, Daniel Farkash, Garrett Warnell, Joydeep Biswas, Peter Stone

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Project website: https://hareshkarnan.github.io/sterling/

Journal ref Conference on Robot Learning (CoRL 2023)

GOOD: General Optimization-based Fusion for 3D Object Detection via LiDAR-Camera Object Candidates

Bingqi Shen, Shuwei Dai, Yuyin Chen, Rong Xiong, Yue Wang, Yanmei Jiao

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Uni3D: Exploring Unified 3D Representation at Scale

Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, Xinlong Wang

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Code and Demo: https://github.com/baaivision/Uni3D

Fusion of Satellite Images and Weather Data with Transformer Networks for Downy Mildew Disease Detection

William Maillet, Maryam Ouhami, Adel Hafiane

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Self-supervised Cross-view Representation Reconstruction for Change Captioning

Yunbin Tu, Liang Li, Li Su, Zheng-Jun Zha, Chenggang Yan, Qingming Huang

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICCV 2023

Semantic RGB-D Image Synthesis

Shijie Li, Rong Li, Juergen Gall

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICCV Workshop on Representation Learning with Very Limited Images 2023

BERT4CTR: An Efficient Framework to Combine Pre-trained Language Model with Non-textual Features for CTR Prediction

Dong Wang, Kavé Salamatian, Yunqing Xia, Weiwei Deng, Qi Zhiang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Tackling Vision Language Tasks Through Learning Inner Monologues

Diji Yang, Kezhen Chen, Jinmeng Rao, Xiaoyuan Guo, Yawen Zhang, Jie Yang, Yi Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Multi-View Fusion and Distillation for Subgrade Distresses Detection based on 3D-GPR

Chunpeng Zhou, Kangjie Ning, Haishuai Wang, Zhi Yu, Sheng Zhou, Jiajun Bu

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field

Haowen Wang, Zhipeng Fan, Zhen Zhao, Zhengping Che, Zhiyuan Xu, Dong Liu, Feifei Feng, Yakun Huang, Xiuquan Qiao, Jian Tang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments The first two authors are with equal contributions. Paper accepted by ACM MM 2023

Relation-Aware Distribution Representation Network for Person Clustering with Multiple Modalities

Kaijian Liu, Shixiang Tang, Ziyue Li, Zhishuai Li, Lei Bai, Feng Zhu, Rui Zhao

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted in IEEE Transactions on Multimedia

Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation

Zunnan Xu, Zhihong Chen, Yong Zhang, Yibing Song, Xiang Wan, Guanbin Li

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Computer Vision and Natural Language Processing. 14 pages, 8 figures. ICCV-2023

HVDetFusion: A Simple and Robust Camera-Radar Fusion Framework

Kai Lei, Zhan Chen, Shuman Jia, Xiaoteng Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

ROFusion: Efficient Object Detection using Hybrid Point-wise Radar-Optical Fusion

Liu Liu, Shuaifeng Zhi, Zhenhua Du, Li Liu, Xinyu Zhang, Kai Huo, Weidong Jiang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Parameter-Efficient Abstractive Question Answering over Tables or Text

Vaishali Pal, Evangelos Kanoulas, Maarten de Rijke

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Published in Dialdoc Workshop at ACL 2022

Hierarchical Graph Neural Networks for Proprioceptive 6D Pose Estimation of In-hand Objects

Alireza Rezazadeh, Snehal Dikhale, Soshi Iba, Nawid Jamali

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

WiCo: Win-win Cooperation of Bottom-up and Top-down Referring Image Segmentation

Zesen Cheng, Peng Jin, Hao Li, Kehan Li, Siheng Li, Xiangyang Ji, Chang Liu, Jie Chen

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted to IJCAI2023

Trustworthy Sensor Fusion against Inaudible Command Attacks in Advanced Driver-Assistance System

Jiwei Guan, Lei Pan, Chen Wang, Shui Yu, Longxiang Gao, Xi Zheng

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI、eess.AS

DocFormerv2: Local Features for Document Understanding

Srikar Appalaraju, Peng Tang, Qi Dong, Nishant Sankaran, Yichu Zhou, R. Manmatha

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.CL

ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning

Xiao Xu, Bei Li, Chenfei Wu, Shao-Yen Tseng, Anahita Bhiwandiwalla, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ACL 2023 Main Conference, Oral

Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understanding

Mingliang Zhai, Yulin Li, Xiameng Qin, Chen Yi, Qunyi Xie, Chengquan Zhang, Kun Yao, Yuwei Wu, Yunde Jia

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments IJCAI 2023

Emergent Communication with Attention

Ryokan Ri, Ryo Ueda, Jason Naradowsky

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CL、cs.AI

Comments CogSci 2023