A fundamental question in cognitive science and AI concerns whether different learning modalities: language, vision, and action, give rise to distinct or shared internal representations. Traditional views assume that models trained on different data types develop specialized, non-transferable representations. However, recent evidence suggests unexpected convergence: models optimized for distinct tasks may develop similar representational geometries. We investigate whether this convergence extends to embodied action learning by training a transformer-based agent to execute goal-directed behaviors in response to natural language instructions. Using behavioral cloning on the BabyAI platform, we generated action-grounded language embeddings shaped exclusively by sensorimotor control requirements. We then compared these representations with those extracted from state-of-the-art large language models (LLaMA, Qwen, DeepSeek, BERT) and vision-language models (CLIP, BLIP). Despite substantial differences in training data, modality, and objectives, we observed robust cross-modal alignment. Action representations aligned strongly with decoder-only language models and BLIP (precision@15: 0.70-0.73), approaching the alignment observed among language models themselves. Alignment with CLIP and BERT was significantly weaker. These findings indicate that linguistic, visual, and action representations converge toward partially shared semantic structures, supporting modality-independent semantic organization and highlighting potential for cross-domain transfer in embodied AI systems.
Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models
School of Informatics, Xiamen University(厦门大学信息学院);Center for Smart Health, School of Nursing, The Hong Kong Polytechnic University(香港理工大学护理学院智能健康中心);Department of Biomedical Informatics, Harvard Medical School(哈佛医学院生物医学信息学系);Department of Radiology, Zhongshan Hospital (Xiamen), Fudan University(复旦大学中山医院放射科);National Institute for Data Science in Health and Medicine(医学与数据科学国家研究院);Department of Computer Science, School of Informatics, Xiamen University(厦门大学信息学院计算机科学系)
Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models
Department of Computer Science and Engineering, The Ohio State University(计算机科学与工程系,俄亥俄州立大学);Translational Data Analytics Institute, The Ohio State University(转化数据分析研究所,俄亥俄州立大学);Department of Biomedical Informatics, The Ohio State University(生物医学信息学系,俄亥俄州立大学)
Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Peng Cheng Laboratory(鹏城实验室);Wuhan AI Research(武汉人工智能研究院)
TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing(北京未来区块链与隐私计算先进创新中心);Beihang University(北京航空航天大学);Hangzhou International Innovation Institute(杭州国际创新研究院);Xreal;Renmin University(中国人民大学);Peking University(北京大学);Beijing Academy of Blockchain and Edge Computing (BABEC)(北京区块链与边缘计算研究院)
Stream-Omni: Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS)(中国科学院智能信息处理重点实验室);Key Laboratory of AI Safety, Chinese Academy of Sciences(中国科学院人工智能安全重点实验室);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学)
Distributional Vision-Language Alignment by Cauchy-Schwarz Divergence
University of Amsterdam(阿姆斯特丹大学);The Netherlands Cancer Institute(荷兰癌症研究所);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);The Arctic University of Norway(挪威北极大学);Singapore Management University(新加坡管理大学)
Decomposed Vision-Language Alignment for Fine-Grained Open-Vocabulary Segmentation
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院航空航天信息研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhejiang University(浙江大学)
VLA-Mark: A cross modal watermark for large vision-language alignment model
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);University of Toronto(多伦多大学);Ant Group, Alibaba(蚂蚁集团,阿里巴巴);New York University Shanghai(纽约大学上海分校)
Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers
Fraunhofer Heinrich-Hertz-Institute(弗劳恩霍夫海因茨·赫兹研究所);Technische Universität Berlin(柏林工业大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所(BIFOLD));Carl von Ossietzky Universität Oldenburg(卡尔·冯·奥西茨基奥尔登堡大学)