多模态大模型

Lost and Found: Overcoming Detector Failures in Online Multi-Object Tracking

Lorenzo Vaquero, Yihong Xu, Xavier Alameda-Pineda, Victor M. Brea, Manuel Mucientes

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted at ECCV 2024. Code available at https://github.com/lorenzovaquero/BUSCA

ProtDAT: A Unified Framework for Protein Sequence Design from Any Protein Text Description

Xiao-Yu Guo, Yi-Fan Li, Yuan Liu, Xiaoyong Pan, Hong-Bin Shen

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Construction and optimization of health behavior prediction model for the elderly in smart elderly care

Qian Guo, Peiyuan Chen

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Comments 23 pages

Towards Unified Molecule-Enhanced Pathology Image Representation Learning via Integrating Spatial Transcriptomics

Minghao Han, Dingkang Yang, Jiabei Cheng, Xukun Zhang, Linhao Qu, Zizhi Chen, Lihua Zhang

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments 21 pages, 11 figures, 7 tables

ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models

Xubing Ye, Yukang Gan, Yixiao Ge, Xiao-Ping Zhang, Yansong Tang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments 11 pages, 4 figures

QuantAttack: Exploiting Dynamic Quantization to Attack Vision Transformers

Amit Baras, Alon Zolfi, Yuval Elovici, Asaf Shabtai

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

ΩSFormer: Dual-Modal Ω-like Super-Resolution Transformer Network for Cross-scale and High-accuracy Terraced Field Vectorization Extraction

Chang Li, Yu Wang, Ce Zhang, Yongjun Zhang

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Blockchain Meets LLMs: A Living Survey on Bidirectional Integration

Jianghao Gong, Peiqi Yan, Yue Zhang, Hongli An, Logan Liu

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

EnStack: An Ensemble Stacking Framework of Large Language Models for Enhanced Vulnerability Detection in Source Code

Shahriyar Zaman Ridoy, Md. Shazzad Hossain Shaon, Alfredo Cuzzocrea, Mst Shapna Akter

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CL

Comments Accepted in 2024 IEEE International Conference on Big Data (IEEE BigData 2024)

Modality Alignment Meets Federated Broadcasting

Yuting Ma, Shengeng Tang, Xiaohua Xu, Lechao Cheng

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

MolMetaLM: a Physicochemical Knowledge-Guided Molecular Meta Language Model

Yifan Wu, Min Zeng, Yang Li, Yang Zhang, Min Li

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CL

Grid and Road Expressions Are Complementary for Trajectory Representation Learning

Silin Zhou, Shuo Shang, Lisi Chen, Peng Han, Christian S. Jensen

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Comments This paper is accepted by KDD2025(August Cycle)

FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression

Yuke Zhu, Chi Xie, Shuang Liang, Bo Zheng, Sheng Guo

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

ImOV3D: Learning Open-Vocabulary Point Clouds 3D Object Detection from Only 2D Images

Timing Yang, Yuanliang Ju, Li Yi

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024. Code link https://github.com/yangtiming/ImOV3D

Journal ref NeurIPS 2024

DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model

Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee. K. Wong, Zhenguo Li, Hengshuang Zhao

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by RA-L. The project page is available at https://tonyxuqaq.github.io/projects/DriveGPT4/

Hierarchical Visual Feature Aggregation for OCR-Free Document Understanding

Jaeyoo Park, Jin Young Choi, Jeonghyung Park, Bohyung Han

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments NeurIPS 2024

Implicit Neural Representations for Simultaneous Reduction and Continuous Reconstruction of Multi-Altitude Climate Data

Alif Bin Abdul Qayyum, Xihaier Luo, Nathan M. Urban, Xiaoning Qian, Byung-Jun Yoon

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2401.16936

Multi-label Cluster Discrimination for Visual Representation Learning

Xiang An, Kaicheng Yang, Xiangzi Dai, Ziyong Feng, Jiankang Deng

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CV

Comments Accepted by ECCV2024

Precise Drive with VLM: First Prize Solution for PRCV 2024 Drive LM challenge

Bin Huang, Siyu Wang, Yuanpeng Chen, Yidan Wu, Hui Song, Zifan Ding, Jing Leng, Chengpeng Liang, Peng Xue, Junliang Zhang, Tiankun Zhao

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Predictive Dynamic Fusion

Bing Cao, Yinan Xia, Yi Ding, Changqing Zhang, Qinghua Hu

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by ICML 2024

MS-DETR: Multispectral Pedestrian Detection Transformer with Loosely Coupled Fusion and Modality-Balanced Optimization

Yinghui Xing, Shuo Yang, Song Wang, Shizhou Zhang, Guoqiang Liang, Xiuwei Zhang, Yanning Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments The paper has been accepted by IEEE Transactions on Intelligent Transportation Systems

Journal ref IEEE Transactions on Intelligent Transportation Systems 2024

PDSR: Efficient UAV Deployment for Swift and Accurate Post-Disaster Search and Rescue

Alaa Awad Abdellatif, Ali Elmancy, Amr Mohamed, Ahmed Massoud, Wadha Lebda, Khalid K. Naji

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Comments This paper is currently under review at IEEE IoT Magazine

EG-SpikeFormer: Eye-Gaze Guided Transformer on Spiking Neural Networks for Medical Image Analysis

Yi Pan, Hanqi Jiang, Junhao Chen, Yiwei Li, Huaqin Zhao, Yifan Zhou, Peng Shu, Zihao Wu, Zhengliang Liu, Dajiang Zhu, Xiang Li, Yohannes Abate, Tianming Liu

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities

Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, Arsalan Shahid

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CL

Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey

Kun Shi, Shibo He, Zhenyu Shi, Anjun Chen, Zehui Xiong, Jiming Chen, Jun Luo

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Joint Embeddings for Graph Instruction Tuning

Aaron Haag, Vlad Argatu, Oliver Lohse

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CL

Comments This work has been submitted to the IEEE for possible publication

CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model

Cheng Chen, Junchen Zhu, Xu Luo, Hengtao Shen, Lianli Gao, Jingkuan Song

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection

Chuhong Mai, Ro-ee Tal, Thahir Mohamed

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CL

Comments Accepted to NeurIPS 2024 Table Representation Learning workshop

Context-Enhanced Multi-View Trajectory Representation Learning: Bridging the Gap through Self-Supervised Models

Tangwen Qian, Junhe Li, Yile Chen, Gao Cong, Tao Sun, Fei Wang, Yongjun Xu

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.AI

scFusionTTT: Single-cell transcriptomics and proteomics fusion with Test-Time Training layers

Dian Meng, Bohao Xing, Xinlei Huang, Yanran Liu, Yijun Zhou, Yongjun xiao, Zitong Yu, Xubin Zheng

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI