Grounded Answers for Multi-agent Decision-making Problem through Generative World Model
National Engineering Research Center for Visual Information and Application(国家视觉信息与应用工程研究中心);Institute of Artificial Intelligence and Robotics(人工智能与机器人研究所);Xi’an Jiaotong University(西安交通大学)
Zeyang Liu, Xinrui Yang, Shiguang Sun, Long Qian, Lipeng Wan, Xingyu Chen, Xuguang Lan
Recent progress in generative models has stimulated significant innovations in many fields, such as image generation and chatbots. Despite their success, these models often produce sketchy and misleading solutions for complex multi-agent decision-making problems because they miss the trial-and-error experience and reasoning as humans. To address this limitation, we explore a paradigm that integrates a language-guided simulator into the multi-agent reinforcement learning pipeline to enhance the generated answer. The simulator is a world model that separately learns dynamics and reward, where the dynamics model comprises an image tokenizer as well as a causal transformer to generate interaction transitions autoregressively, and the reward model is a bidirectional transformer learned by maximizing the likelihood of trajectories in the expert demonstrations under language guidance. Given an image of the current state and the task description, we use the world model to train the joint policy and produce the image sequence as the answer by running the converged policy on the dynamics model. The empirical results demonstrate that this framework can improve the answers for multi-agent decision-making problems by showing superior performance on the training and unseen tasks of the StarCraft Multi-Agent Challenge benchmark. In particular, it can generate consistent interaction sequences and explainable reward functions at interaction states, opening the path for training generative models of the future.
MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering
Lanzhou University(兰州大学);National University of Singapore(新加坡国立大学);Sun Yat-sen University(中山大学);Jinan University(暨南大学);Nanyang Technological University(南洋理工大学);Peking University(北京大学);University of Science and Technology of China(中国科学技术大学);Hefei University of Technology(合肥工业大学)
DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning
Department of Automation, Tsinghua University, Beijing, China(清华大学自动化系);Zhongguancun Academy, Beijing, China(中关村学院);School of Software, Xinjiang University, Urumqi, China(新疆大学软件学院);College of Materials Science and Engineering, Fuzhou University, Fuzhou, China(福州大学材料科学与工程学院);Institute of Automation, Chinese Academy of Sciences, Beijing, China(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China(中国科学院大学人工智能学院);Beijing Qianjue Technology Co., Ltd., Beijing, China(北京千 jue 技术有限公司)
FinRank: An Evidence-Grounded Benchmark for Financial Question Answering and Retrieval over SEC Filings
University of Groningen(格罗宁根大学);Goethe University Frankfurt(法兰克福大学);DataNXT GmbH(DataNXT有限公司);Halle Institute for Economic Research (IWH)(哈雷经济研究所);Martin Luther University Halle-Wittenberg(马丁路德大学哈雷-维滕贝格分校)
The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions
Department of Computer Science, University College London, The United Kingdom(伦敦大学学院计算机科学系,英国);Centre for Artificial Intelligence, University College London, The United Kingdom(伦敦大学学院人工智能中心,英国)
HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways
University of Calabria(卡拉布里亚大学);University of Catanzaro(卡坦扎罗大学);University of Eastern Piedmont(东皮埃蒙特大学);University of Pisa(比萨大学);University of Cambridge(剑桥大学)
CausalChaos! Dataset for Comprehensive Causal Action Question Answering Over Longer Causal Chains Grounded in Dynamic Visual Scenes
Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore(高性能计算研究所,科技研究局,新加坡);Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学);Singapore Polytechnic(新加坡理工学院)
EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries
KAIST(韩国科学技术院);Seoul National University(首尔大学);Seoul National University Bundang Hospital(首尔大学盆唐医院);SAIHST, Sungkyunkwan University(成均馆大学);Yonsei University College of Medicine(延世大学医学院);Gangnam Severance Hospital(江南塞弗伦斯医院);Severance Hospital(塞弗伦斯医院);Seoul Medical Center(首尔医疗中心);Seoul National University Hospital(首尔大学医院);National Cancer Center(国立癌症中心);Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院);Samsung Medical Center(三星医疗中心)