Building joint representations across images and text is an essential step for tasks such as Visual Question Answering and Video Question Answering. In this work, we find that the representations must not only jointly capture features from both modalities but should also be diverse for better generalization performance. To this end, we propose joint vision-language representation learning by diversifying the tokenization learning process, enabling tokens that are sufficiently disentangled from each other to be learned from both modalities. We observe that our approach outperforms the baseline models in a majority of settings and is competitive with state-of-the-art methods.
Parameter-Efficient CLIP Adaptation for 3D Understanding via Unified Tokenization
Fondazione Bruno Kessler（布鲁诺·科塞拉基金会）；University of Trento（特伦托大学）；University of Pisa（比萨大学）；Beijing Forestry University（北京林业大学）；Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology (GZ)（香港科技大学）；Shandong University（山东大学）；University of California, Merced（加州大学默塞德分校）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Dalian University of Technology（大连理工大学）；Tsinghua University（清华大学）
针对视觉分词器与下游任务优化脱节导致的表示瓶颈，提出 ETT 端到端联合调优方法，在多模态理解与生成任务上较冻结基线提升 2-6%。
BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization
University of Science and Technology of China (USTC)（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Huawei Inc.（华为公司）
BioGait-VLM: A Tri-Modal Vision-Language-Biomechanics Framework for Interpretable Clinical Gait Assessment
Department of Computer Science, Drexel University（德克萨斯大学计算机科学系）；Department of Neurological Surgery, Washington University（华盛顿大学神经外科系）；University of California, Berkeley（加州大学伯克利分校）
HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models
Simula Metropolitan Center for Digital Engineering (SimulaMet)（Simula数字工程研究中心）；Oslo Metropolitan University (OsloMet)（奥斯陆 Metropolitan 大学）；Simula Research Laboratory（Simula研究实验室）
ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models
Huazhong University of Science and Technology（华中科技大学）；Zhongguancun Academy（中关村学院）；DeepCybo；Harbin Institute of Technology（哈尔滨工业大学）；Beihang University（北京航空航天大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Zhengzhou University（郑州大学）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）
Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation
Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea（韩国首尔峨山医疗中心蔚山大学医学院融合医学系）；University of Ulsan College of Medicine, Seoul, Republic of Korea（韩国首尔蔚山大学医学院）；Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea（韩国首尔峨山医疗中心蔚山大学医学院放射科与放射学研究所）
FLM: Frequency-Aware Language Models for Generative Image Compression
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；University of Science and Technology of China（中国科学技术大学）；College of Intelligent Systems Science and Engineering, Harbin Engineering University（哈尔滨工程大学智能系统科学与工程学院）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局信息通信研究院）；National Tsing Hua University（国立清华大学）
TMT: Tri-Modal Translation between Speech, Image, and Text by Processing Different Modalities as Different Languages
Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)（整合视觉与语言实验室，电气工程学院，韩国科学技术院（KAIST））；Language Technologies Institute, Carnegie Mellon University（语言技术研究所，卡内基梅隆大学）
WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehouses
Center for AI Research, VinUniversity（VinUniversity人工智能研究中心）；College of Engineering and Computer Science, VinUniversity（VinUniversity工程与计算机科学学院）；VinRobotics（VinRobotics公司）
