多模态大模型

Test-Time Scaling Strategies for Generative Retrieval in Multimodal Conversational Recommendations

Hung-Chun Hsu, Yuan-Ching Kuo, Chao-Han Huck Yang, Szu-Wei Fu, Hanrong Ye, Hongxu Yin, Yu-Chiang Frank Wang, Ming-Feng Tsai, Chuan-Ju Wang

机构 * Research Center for Information Technology Innovation, Academia Sinica（资讯科技创新研究所以） ； NVIDIA（NVIDIA公司） ； Department of Computer Science, National Chengchi University（国立政治大学计算机科学系） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environments

Dmitry Yudin

机构 * Moscow Institute of Physics and Technology（莫斯科物理技术学院） ； AIRI 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments 29 pages, 3 figures, 13 tables. Preprint of the accepted article in Optical Memory and Neural Network Journal

SoccerRAG: Multimodal Soccer Information Retrieval via Natural Queries

Aleksander Theo Strand, Sushant Gautam, Cise Midoglu, Pål Halvorsen

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.MM

Comments accepted to CBMI 2024 as a regular paper; https://github.com/simula/soccer-rag

Multimodal Medical Endoscopic Image Analysis via Progressive Disentangle-aware Contrastive Learning

Junhao Wu, Yun Li, Junhao Li, Jingliang Bian, Xiaomao Fan, Wenbin Lei, Ruxin Wang

机构 * Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院） ； First Affiliated Hospital, Sun Yat-sen University（中山大学第一附属医院） ； College of Big Data and Internet, Shenzhen Technology University（深圳技术大学大数据与互联网学院） 纠错

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments 12 pages,6 figures, 6 tables

RT-Cache: Training-Free Retrieval for Real-Time Manipulation

Owen Kwon, Abraham George, Alison Bartsch, Amir Barati Farimani

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 6 figures. 2025 IEEE-RAS 24th International Conference on Humanoid Robots

Does GPT-4 surpass human performance in linguistic pragmatics?

Ljubisa Bojic, Predrag Kovacevic, Milan Cabarkapa

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 19 pages, 1 figure, 2 tables

Consciousness as a Functor

Sridhar Mahadevan

机构 * Adobe Research and University of Massachusetts, Amherst（Adobe研究院和马萨诸塞大学） 纠错

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.AI

Comments 31 pages

VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding

Jian Chen, Ming Li, Jihyung Kil, Chenguang Wang, Tong Yu, Ryan Rossi, Tianyi Zhou, Changyou Chen, Ruiyi Zhang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Under Review

Demo: Soccer Information Retrieval via Natural Queries using SoccerRAG

Aleksander Theo Strand, Sushant Gautam, Cise Midoglu, Pål Halvorsen

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.MM

Comments accepted to CBMI 2024 as a demonstration; https://github.com/simula/soccer-rag. arXiv admin note: text overlap with arXiv:2406.01273

LGE-Guided Cross-Modality Contrastive Learning for Gadolinium-Free Cardiomyopathy Screening in Cine CMR

Siqing Yuan, Yulin Wang, Zirui Cao, Yueyan Wang, Zehao Weng, Hui Wang, Lei Xu, Zixian Chen, Lei Chen, Zhong Xue, Dinggang Shen

机构 * School of Biomedical Engineering（生物医学工程学院） ； State Key Laboratory of Advanced Medical Materials and Devices（先进医疗材料与设备国家重点实验室） ； ShanghaiTech University（上海科技大学） ； United Imaging Intelligence（联合影像智能） ； Shanghai Clinical Research and Trial Center（上海临床研究与试验中心） ； Capital Medical University（首都医科大学） ； The First Hospital of Lanzhou University（兰州大学第一医院） 纠错

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted to MLMI 2025 (MICCAI workshop); camera-ready version

Sparse and Dense Retrievers Learn Better Together: Joint Sparse-Dense Optimization for Text-Image Retrieval

Jonghyun Song, Youngjune Lee, Gyu-Hwung Cho, Ilhyeon Song, Saehun Kim, Yohan Jo

机构 * Graduate School of Data Science, Seoul National University（首尔国立大学数据科学研究生院） ； NAVER Corporation（NAVER公司） 纠错

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

Comments accepted to CIKM 2025 short research paper track

ShaLa: Multimodal Shared Latent Space Modelling

Jiali Cui, Yan-Ying Chen, Yanxia Zhang, Matthew Klenk

专题命中 多模态生成 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

HunyuanVideo-Foley: Multimodal Diffusion with Representation Alignment for High-Fidelity Foley Audio Generation

Sizhe Shan, Qiulin Li, Yutao Cui, Miles Yang, Yuehai Wang, Qun Yang, Jin Zhou, Zhao Zhong

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、eess.AS

One Framework to Rule Them All: Unifying Multimodal Tasks with LLM Neural-Tuning

Hao Sun, Yu Song, Jiaqing Liu, Jihong Hu, Yen-Wei Chen, Lanfen Lin

机构 * College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院） ； College of Information Science and Engineering, Ritsumeikan University（立命馆大学信息科学与工程学院） 纠错

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-on

Aowen Wang, Wei Li, Hao Luo, Mengxing Ao, Chenyu Zhu, Xinyang Li, Fan Wang

机构 * DAMO Academy, Alibaba Group（达摩院，阿里巴巴集团） ； Hupan Lab（汇安实验室） ； Zhejiang University（浙江大学） 纠错

专题命中 多模态生成 ：multi-modal(title,abstract)；分类 cs.CV

MMCIG: Multimodal Cover Image Generation for Text-only Documents and Its Dataset Construction via Pseudo-labeling

Hyeyeon Kim, Sungwoo Han, Jingun Kwon, Hidetaka Kamigaito, Manabu Okumura

机构 * Chungnam National University（Chungnam 国立大学） ； Nara Institute of Science and Technology (NAIST)（Nara 科学技术研究所） ； Institute of Science Tokyo（东京科学研究所） 纠错

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV

WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation

Rabiul Awal, Mahsa Massoud, Aarash Feizi, Zichao Li, Suyuchen Wang, Christopher Pal, Aishwarya Agrawal, David Vazquez, Siva Reddy, Juan A. Rodriguez, Perouz Taslakian, Spandana Gella, Sai Rajeswar

机构 * ServiceNow ； Mila ； Université de Montréal（蒙特利尔大学） ； McGill University（麦吉尔大学） ； École de Technologie Supérieure (ETS)（高等技术学院） ； Polytechnique Montréal（蒙特利尔理工学院） 纠错

专题命中 多模态生成 ：multimodal(title,abstract)；分类 cs.CV

Comments This paper has been accepted to the EMNLP 2025 main conference. Check the project page here: https://webmmu-paper.github.io/

Instant Preference Alignment for Text-to-Image Diffusion Models

Yang Li, Songlin Yang, Xiaoxuan Han, Wei Wang, Jing Dong, Yueming Lyu, Ziyu Xue

机构 * New Laboratory of Pattern Recognition, CASIA（模式识别新实验室，中国科学院自动化研究所） ； The Hong Kong University of Science and Technology（香港科技大学） ； Nanjing university（南京大学） ； Academy of Broadcasting Science, NRTA（广播科学研究院，国家广播电视总局） 纠错

专题命中 多模态生成 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 17 figures

DanceEditor: Towards Iterative Editable Music-driven Dance Generation with Open-Vocabulary Descriptions

Hengyuan Zhang, Zhe Li, Xingqun Qi, Mengze Li, Muyi Sun, Man Zhang, Sirui Han

机构 * Peking University（北京大学） ； The Hong Kong University of Science and Technology（香港科技大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Journal ref ICCV 2025

SSG-Dit: A Spatial Signal Guided Framework for Controllable Video Generation

Peng Hu, Yu Gu, Liang Luo, Fuji Ren

机构 * School of Computer Science and Engineering, University of Electronic Science and Technology of China（计算机科学与工程学院，电子科技大学） 纠错

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance

Ayce Idil Aytekin, Helge Rhodin, Rishabh Dabral, Christian Theobalt

机构 * Max Planck Institute for Informatics and Saarland University（马克斯·普朗克研究所（信息学）和萨尔兰大学） ； Bielefeld University（比勒菲尔德大学） 纠错

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments Project page: https://aidilayce.github.io/FollowMyHold-page/

V2X-R: Cooperative LiDAR-4D Radar Fusion with Denoising Diffusion for 3D Object Detection

Xun Huang, Jinlong Wang, Qiming Xia, Siheng Chen, Bisheng Yang, Xin Li, Cheng Wang, Chenglu Wen

机构 * Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University, China（福建智能城市感知与计算重点实验室，厦门大学） ； Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China（多媒体可信感知与高效计算重点实验室，中国教育部，厦门大学） ； Zhongguancun Academy（中关村学院） ； Shanghai Jiao Tong University（上海交通大学） ； Wuhan University（武汉大学） ； Texas A&M University（德克萨斯A&M大学） 纠错

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR2025

From Classical Probabilistic Latent Variable Models to Modern Generative AI: A Unified Perspective

Tianhua Chen

机构 * School of Computing and Engineering University of Huddersfield（计算与工程学院赫德斯菲尔德大学） 纠错

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.AI

Comments This is a substantially improved and expanded version of an earlier manuscript hosted on SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5244929

DiffS-NOCS: 3D Point Cloud Reconstruction through Coloring Sketches to NOCS Maps Using Diffusion Models

Di Kong, Qianhui Wan

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； Tsinghua University（清华大学） ； Zhongguancun Academy（中关村学院） ； Beijing Normal University（北京师范大学） 纠错

专题命中 多模态生成 ：multi-modal(abstract)；分类 cs.CV

Generative Machine Learning in Adaptive Control of Dynamic Manufacturing Processes: A Review

Suk Ki Lee, Hyunwoong Ko

机构 * School of Manufacturing Systems and Networks, Arizona State University（制造系统与网络学院，亚利桑那州立大学） 纠错

专题命中 多模态生成 ：multimodal(abstract)

Comments 12 pages, 1 figure, 1 table. This paper has been accepted for publication in the proceedings of ASME IDETC-CIE 2025

Intern-S1: A Scientific Multimodal Foundation Model

Lei Bai, Zhongrui Cai, Yuhang Cao, Maosong Cao, Weihan Cao, Chiyu Chen, Haojiong Chen, Kai Chen, Pengcheng Chen, Ying Chen, Yongkang Chen, Yu Cheng, Pei Chu, Tao Chu, Erfei Cui, Ganqu Cui, Long Cui, Ziyun Cui, Nianchen Deng, Ning Ding, Nanqing Dong, Peijie Dong, Shihan Dou, Sinan Du, Haodong Duan, Caihua Fan, Ben Gao, Changjiang Gao, Jianfei Gao, Songyang Gao, Yang Gao, Zhangwei Gao, Jiaye Ge, Qiming Ge, Lixin Gu, Yuzhe Gu, Aijia Guo, Qipeng Guo, Xu Guo, Conghui He, Junjun He, Yili Hong, Siyuan Hou, Caiyu Hu, Hanglei Hu, Jucheng Hu, Ming Hu, Zhouqi Hua, Haian Huang, Junhao Huang, Xu Huang, Zixian Huang, Zhe Jiang, Lingkai Kong, Linyang Li, Peiji Li, Pengze Li, Shuaibin Li, Tianbin Li, Wei Li, Yuqiang Li, Dahua Lin, Junyao Lin, Tianyi Lin, Zhishan Lin, Hongwei Liu, Jiangning Liu, Jiyao Liu, Junnan Liu, Kai Liu, Kaiwen Liu, Kuikun Liu, Shichun Liu, Shudong Liu, Wei Liu, Xinyao Liu, Yuhong Liu, Zhan Liu, Yinquan Lu, Haijun Lv, Hongxia Lv, Huijie Lv, Qitan Lv, Ying Lv, Chengqi Lyu, Chenglong Ma, Jianpeng Ma, Ren Ma, Runmin Ma, Runyuan Ma, Xinzhu Ma, Yichuan Ma, Zihan Ma, Sixuan Mi, Junzhi Ning, Wenchang Ning, Xinle Pang, Jiahui Peng, Runyu Peng, Yu Qiao, Jiantao Qiu, Xiaoye Qu, Yuan Qu, Yuchen Ren, Fukai Shang, Wenqi Shao, Junhao Shen, Shuaike Shen, Chunfeng Song, Demin Song, Diping Song, Chenlin Su, Weijie Su, Weigao Sun, Yu Sun, Qian Tan, Cheng Tang, Huanze Tang, Kexian Tang, Shixiang Tang, Jian Tong, Aoran Wang, Bin Wang, Dong Wang, Lintao Wang, Rui Wang, Weiyun Wang, Wenhai Wang, Jiaqi Wang, Yi Wang, Ziyi Wang, Ling-I Wu, Wen Wu, Yue Wu, Zijian Wu, Linchen Xiao, Shuhao Xing, Chao Xu, Huihui Xu, Jun Xu, Ruiliang Xu, Wanghan Xu, GanLin Yang, Yuming Yang, Haochen Ye, Jin Ye, Shenglong Ye, Jia Yu, Jiashuo Yu, Jing Yu, Fei Yuan, Yuhang Zang, Bo Zhang, Chao Zhang, Chen Zhang, Hongjie Zhang, Jin Zhang, Qiaosheng Zhang, Qiuyinzhe Zhang, Songyang Zhang, Taolin Zhang, Wenlong Zhang, Wenwei Zhang, Yechen Zhang, Ziyang Zhang, Haiteng Zhao, Qian Zhao, Xiangyu Zhao, Xiangyu Zhao, Bowen Zhou, Dongzhan Zhou, Peiheng Zhou, Yuhao Zhou, Yunhua Zhou, Dongsheng Zhu, Lin Zhu, Yicheng Zou

机构 * Intern-S1 Team（Intern-S1团队） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；multimodal foundation model(title)；分类 cs.CV、cs.CL

Traveling Across Languages: Benchmarking Cross-Lingual Consistency in Multimodal LLMs

Hao Wang, Pinzhi Huang, Jihan Yang, Saining Xie, Daisuke Kawahara

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments The first version of this paper mistakenly included a prompt injection phrase, which was inappropriate and unprofessional. Although we corrected the version on arXiv and withdrew from the conference, my co-authors and university strongly request a full withdrawal. Given the situation, I no longer have the authority to manage this paper, and withdrawing it from arXiv is the most responsible action

Egocentric Instruction-oriented Affordance Prediction via Large Multimodal Model

Bokai Ji, Jie Gu, Xiaokang Ma, Chu Tang, Jingmin Chen, Guangxia Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Hidden in Plain Sight: Reasoning in Underspecified and Misspecified Scenarios for Multimodal LLMs

Qianqi Yan, Hongquan Li, Shan Jiang, Yang Zhao, Xinze Guan, Ching-Chen Kuo, Xin Eric Wang

机构 * University of California, Santa Cruz（加州大学圣克鲁兹分校） ； eBay 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

A Multimodal Handover Failure Detection Dataset and Baselines

Santosh Thoduka, Nico Hochgeschwender, Juergen Gall, Paul G. Plöger

机构 * Hochschule Bonn-Rhein-Sieg（波恩-莱茵-锡格应用科学大学） ； University of Bremen（不莱梅大学） ； University of Bonn（波恩大学） ； Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔人工智能与机器学习研究所） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at ICRA 2024