World models are increasingly pivotal in interpreting and simulating the rules and actions of complex environments. Genie, a recent model, excels at learning from visually diverse environments but relies on costly human-collected data. We observe that their alternative method of using random agents is too limited to explore the environment. We propose to improve the model by employing reinforcement learning based agents for data generation. This approach produces diverse datasets that enhance the model's ability to adapt and perform well across various scenarios and realistic actions within the environment. In this paper, we first release the model GenieRedux - an implementation based on Genie. Additionally, we introduce GenieRedux-G, a variant that uses the agent's readily available actions to factor out action prediction uncertainty during validation. Our evaluation, including a replication of the Coinrun case study, shows that GenieRedux-G achieves superior visual fidelity and controllability using the trained agent exploration. The proposed approach is reproducable, scalable and adaptable to new types of environments. Our codebase is available at https://github.com/insait-institute/GenieRedux .
MagicWorld: Towards Long-Horizon Stability for Interactive Video World Exploration
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);vivo BlueImage Lab, vivo Mobile Communication Co., Ltd.(vivo蓝影实验室,vivo移动通信有限公司);National University of Singapore(新加坡国立大学)
Sensorium Arc: AI Agent System for Oceanic Data Exploration and Interactive Eco-Art
Immersive Media Design University of Maryland College Park(沉浸媒体设计大学马里兰大学学院公园分校);Center for the Study of the Force Majeure University of California, Santa Cruz(重大研究机构加州大学圣克鲁兹分校);Virtual Planet Technologies(虚拟星球技术)
RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation
Peking University(北京大学);National University of Singapore(新加坡国立大学);Columbia University(哥伦比亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Amazon(亚马逊公司)
Perceive, Reflect and Understand Long Video: Progressive Multi-Granular Clue Exploration with Interactive Agents
School of Electronic Engineering, Xidian University(西安电子科技大学电子工程学院);Department of Computer Science and Engineering, Hong Kong University of Science and Technology(香港理工大学计算机科学与工程系);College of Computer and Information, Hohai University(河海大学计算机与信息学院)
The University of Hong Kong, Pok Fu Lam, Hong Kong(香港大学);Kuaishou Technology, Shenzhen, China(快手科技);The Hong Kong University of Science and Technology (HKUST), Hong Kong(香港科技大学)
Compressing Human Body Video with Interactive Semantics: A Generative Approach
City University of Hong Kong(香港城市大学);Alibaba DAMO Academy & Hupan Laboratory(阿里达摩院及胡畔实验室);Shenzhen Research Institute, City University of Hong Kong(香港城市大学深圳研究院)
Interactive Face Video Coding: A Generative Compression Framework
Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系);School of Computer Science, Peking University(北京大学计算机学院);DAMO Academy, Alibaba Group(阿里巴巴集团 DAMO 院);Shenzhen Research Institute, City University of Hong Kong(香港城市大学深圳研究学院)
Spacewalker: Traversing Representation Spaces for Fast Interactive Exploration and Annotation of Unstructured Data
Institute for AI in Medicine, University Hospital Essen(埃森大学医院AI医学研究所);Mercedes-Benz Tech Innovation(梅赛德斯-奔驰技术创新公司);Cancer Research Center Cologne Essen, West German Cancer Center Essen, University Hospital Essen(埃森大学医院西德癌症中心科隆-埃森癌症研究中心);Institute of Computer Graphics and Vision, Graz University of Technology(格拉茨工业大学计算机图形与视觉研究所);Department of Physics, TU Dortmund University(多特蒙德工业大学物理系)
Intuitive, Interactive Beard and Hair Synthesis with Generative Models
University of Southern California(南加州大学);USC ICT(南加州大学创新技术研究院);Adobe Inc.(奥多比公司);miHoYo(米哈游);ByteDance Research(字节跳动研究院);Tencent America(腾讯美国);Pinscreen Inc.(Pinscreen公司)