Drawing upon the intuition that aligning different modalities to the same semantic embedding space would allow models to understand states and actions more easily, we propose a new perspective to the offline reinforcement learning (RL) challenge. More concretely, we transform it into a supervised learning task by integrating multimodal and pre-trained language models. Our approach incorporates state information derived from images and action-related data obtained from text, thereby bolstering RL training performance and promoting long-term strategic thinking. We emphasize the contextual understanding of language and demonstrate how decision-making in RL can benefit from aligning states' and actions' representation with languages' representation. Our method significantly outperforms current baselines as evidenced by evaluations conducted on Atari and OpenAI Gym environments. This contributes to advancing offline RL performance and efficiency while providing a novel perspective on offline RL.Our code and data are available at https://github.com/Zheng0428/MORE_.
Coordinating Ride-Pooling with Public Transit using Reward-Guided Conservative Q-Learning: An Offline Training and Online Fine-Tuning Reinforcement Learning Framework
The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Offline-Online Curriculum RL for Multimodal Reasoning
Beijing University of Posts and Telecommunications(北京邮电大学);BUPT Shenzhen Institute(北京邮电大学深圳研究院);Carnegie Mellon University(卡内基梅隆大学);Lancaster University(兰卡斯特大学);Nanyang Technological University(南洋理工大学);China Telecom Corporation Limited Sichuan Branch(中国电信股份有限公司四川分公司);Changsha University of Science & Technology(长沙理工大学)
DRIM: Learning Disentangled Representations from Incomplete Multimodal Healthcare Data
IUCT-Oncopole Institut Claudius Regaud(克劳迪乌斯·雷戈德研究所肿瘤学中心);IRT Saint Exupéry(圣埃克苏佩里技术研究院);INSERM UMR 1037, Cancer Research Center of Toulouse (CRCT)(法国国家健康与医学研究院UMR 1037,图卢兹癌症研究中心)
Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Zhejiang University(浙江大学)
ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptors
Southwest Jiaotong University(西南交通大学);Microsoft Research Asia(微软亚洲研究院);Southwestern University of Finance and Economics(西南财经大学);Ocean University of China(中国海洋大学);Tencent(腾讯)
Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation
School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与工程学院);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算学系);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
Multimodal Medical Image Binding via Shared Text Embeddings
The Hong Kong Polytechnic University(香港理工大学);Emory University(埃默里大学);The University of Hong Kong(香港大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Science and Technology of China(中国科学技术大学);Shanghai AI Laboratory(上海人工智能实验室);Johns Hopkins University(约翰霍普金斯大学)
Texas A&M University(德克萨斯农工大学);Qatar Computing Research Institute(卡塔尔计算研究所);Birzeit University(比尔宰特大学);Hamad Bin Khalifa University(哈马德·本·哈利法大学);University of Toronto(多伦多大学)
EviDep: Uncertainty-Aware Multimodal Depression Estimation via Disentangled Evidential Learning
School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机科学与信息工程学院);Department of Psychiatry, The First Affiliated Hospital of University of Science and Technology of China, Division of Life Sciences and Medicine, University of Science and Technology of China(中国科学技术大学附属第一医院精神科,生命科学与医学学院)
ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content
Qatar Computing Research Institute(卡塔尔计算研究所);HBKU(哈迈德·本·哈利法大学);University of New Brunswick(新不伦瑞克大学);Hamad Bin Khalifa University(哈迈德·本·哈利法大学);Northwestern University in Qatar(西北大学卡塔尔分校)
Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs
School of Computer Science and Engineering, Central South University(中南大学计算机科学与工程学院);Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(香港理工大学数据科学与人工智能系);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Semi-supervised Multimodal Representation Learning through a Global Workspace
CerCo, CNRS UMR 5549, Université de Toulouse and ANITI, Artificial and Natural Intelligence Toulouse Institute(CerCo、CNRS UMR 5549、图卢兹大学和ANITI人工智能与自然智能图卢兹研究所)