多模态大模型

Partial Scene Text Retrieval

Hao Wang, Minghui Liao, Zhouyi Xie, Wenyu Liu, Xiang Bai

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted on TPAMI

SCAN: Bootstrapping Contrastive Pre-training for Data Efficiency

Yangyang Guo, Mohan Kankanhalli

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV

Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosis

Sara Ketabi, Matthias W. Wagner, Cynthia Hawkins, Uri Tabori, Birgit Betina Ertl-Wagner, Farzad Khalvati

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Language-guided Hierarchical Fine-grained Image Forgery Detection and Localization

Xiao Guo, Xiaohong Liu, Iacopo Masi, Xiaoming Liu

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by IJCV2024. arXiv admin note: substantial text overlap with arXiv:2303.17111

AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning

Shirley Wu, Shiyu Zhao, Qian Huang, Kexin Huang, Michihiro Yasunaga, Kaidi Cao, Vassilis N. Ioannidis, Karthik Subbian, Jure Leskovec, James Zou

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

Comments NeurIPS 2024 main conference

LESS: Label-Efficient and Single-Stage Referring 3D Segmentation

Xuexun Liu, Xiaoxu Xu, Jinlong Li, Qiudan Zhang, Xu Wang, Nicu Sebe, Lin Ma

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Knowledge-aware Text-Image Retrieval for Remote Sensing Images

Li Mi, Xianjie Dai, Javiera Castillo-Navarro, Devis Tuia

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by IEEE TGRS

ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text

Han Yu, Peikun Guo, Akane Sano

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

JointMotion: Joint Self-Supervision for Joint Motion Prediction

Royden Wagner, Omer Sahin Tas, Marvin Klemp, Carlos Fernandez

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments CoRL'24 camera-ready

Towards a Reliable Offline Personal AI Assistant for Long Duration Spaceflight

Oliver Bensch, Leonie Bensch, Tommy Nilsson, Florian Saling, Wafa M. Sadri, Carsten Hartmann, Tobias Hecking, J. Nathan Kutz

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments 75th International Astronautical Congress (IAC), Milan, Italy, 14-18 October 2024

RoRA-VLM: Robust Retrieval-Augmented Vision Language Models

Jingyuan Qi, Zhiyang Xu, Rulin Shao, Yang Chen, Jin Di, Yu Cheng, Qifan Wang, Lifu Huang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks

Isaac R. Galatzer-Levy, David Munday, Jed McGiffin, Xin Liu, Danny Karmon, Ilia Labzovsky, Rivka Moroshko, Amir Zait, Daniel McDuff

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Fortify Your Foundations: Practical Privacy and Security for Foundation Model Deployments In The Cloud

Marcin Chrapek, Anjo Vahldiek-Oberwagner, Marcin Spoczynski, Scott Constable, Mona Vij, Torsten Hoefler

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

GS-Hider: Hiding Messages into 3D Gaussian Splatting

Xuanyu Zhang, Jiarui Meng, Runyi Li, Zhipei Xu, Yongbing Zhang, Jian Zhang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024, 3DGS steganography

A Foundation Model for the Solar Dynamics Observatory

James Walsh, Daniel G. Gass, Raul Ramos Pollan, Paul J. Wright, Richard Galvez, Noah Kasmanoff, Jason Naradowsky, Anne Spalding, James Parr, Atılım Güneş Baydin

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

EUFCC-CIR: a Composed Image Retrieval Dataset for GLAM Collections

Francesc Net, Lluis Gomez

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Comments ECCV Workshop (AI4DH2024)

Cacophony: An Improved Contrastive Audio-Text Model

Ge Zhu, Jordan Darefsky, Zhiyao Duan

专题命中 跨模态检索 ：image-text(abstract)；分类 eess.AS

Comments Accepted at IEEE/ACM Transactions on Audio, Speech, and Language Processing

CLSP: High-Fidelity Contrastive Language-State Pre-training for Agent State Representation

Fuxian Huang, Qi Zhang, Shaopeng Zhai, Jie Wang, Tianyi Zhang, Haoran Zhang, Ming Zhou, Yu Liu, Yu Qiao

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Towards Graph Prompt Learning: A Survey and Beyond

Qingqing Long, Yuchen Yan, Peiyan Zhang, Chen Fang, Wentao Cui, Zhiyuan Ning, Meng Xiao, Ning Cao, Xiao Luo, Lingjun Xu, Shiyue Jiang, Zheng Fang, Chong Chen, Xian-Sheng Hua, Yuanchun Zhou

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments I have decided to temporarily withdraw this draft as I am in the process of making further revisions to improve its content

Brain-Cognition Fingerprinting via Graph-GCCA with Contrastive Learning

Yixin Wang, Wei Peng, Yu Zhang, Ehsan Adeli, Qingyu Zhao, Kilian M. Pohl

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Bundle Fragments into a Whole: Mining More Complete Clusters via Submodular Selection of Interesting webpages for Web Topic Detection

Junbiao Pang, Anjing Hu, Qingming Huang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments 10

A Closer Look at the Explainability of Contrastive Language-Image Pre-training

Yi Li, Hualiang Wang, Yiqun Duan, Jiheng Zhang, Xiaomeng Li

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments 30 pages, 11 figures, under review

StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction

Shengkun Wang, Taoran Ji, Linhan Wang, Yanshen Sun, Shang-Ching Liu, Amit Kumar, Chang-Tien Lu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro Astolfi, Kyunghyun Cho, Yann LeCun

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding

Xiaoyu Liang, Jiayuan Yu, Lianrui Mu, Jiedong Zhuang, Jiaqi Hu, Yuchen Yang, Jiangnan Ye, Lu Lu, Jian Chen, Haoji Hu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments PRCV

GS-PT: Exploiting 3D Gaussian Splatting for Comprehensive Point Cloud Understanding via Self-supervised Learning

Keyi Liu, Yeqi Luo, Weidong Yang, Jingyi Xu, Zhijun Li, Wen-Ming Chen, Ben Fei

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

MOSMOS: Multi-organ segmentation facilitated by medical report supervision

Weiwei Tian, Xinyu Huang, Junlin Hou, Caiyue Ren, Longquan Jiang, Rui-Wei Zhao, Gang Jin, Yuejie Zhang, Daoying Geng

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Comments 14 pages, 7 figures

Optimizing CLIP Models for Image Retrieval with Maintained Joint-Embedding Alignment

Konstantin Schall, Kai Uwe Barthel, Nico Hezel, Klaus Jung

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

The Compressor-Retriever Architecture for Language Model OS

Yuan Yang, Siheng Xiong, Ehsan Shareghi, Faramarz Fekri

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery

Long Bai, Guankun Wang, Mobarakol Islam, Lalithkumar Seenivasan, An Wang, Hongliang Ren

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by Information Fusion. Code and data availability: https://github.com/longbai1006/Surgical-VQLAPlus