ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);MIT(麻省理工学院);Stanford University(斯坦福大学);Harvard University(哈佛大学)
Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megum… 展开作者
Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, Kuno Kim, Elias Wang, Michael Lingelbach, Aidan Curtis, Kevin Feigelis, Daniel M. Bear, Dan Gutfreund, David Cox, Antonio Torralba, James J. DiCarlo, Joshua B. Tenenbaum, Josh H. McDermott, Daniel L. K. Yamins
We introduce ThreeDWorld (TDW), a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. Unique properties include: real-time near-photo-realistic image rendering; a library of objects and environments, and routines for their customization; generative procedures for efficiently building classes of new environments; high-fidelity audio rendering; realistic physical interactions for a variety of material types, including cloths, liquid, and deformable objects; customizable agents that embody AI agents; and support for human interactions with VR devices. TDW's API enables multiple agents to interact within a simulation and returns a range of sensor and physics data representing the state of the world. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science, including multi-modal physical scene understanding, physical dynamics predictions, multi-agent interactions, models that learn like a child, and attention studies in humans and neural networks.
Interactive State Space Model with Cross-Modal Local Scanning for Depth Super-Resolution
National University of Defense Technology(国防科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Sun Yat-sen University(中山大学);Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究院(TeleAI));University of Macau(澳门大学)
iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement
Unicom Data Intelligence(中国联通数据智能研究所);Data Science & Artificial Intelligence Research Institute(数据科学与人工智能研究院);China United Network Communications Group Corporation Limited(中国联合网络通信集团有限公司)
PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection
National Key Laboratory for Multimedia Information Processing(多媒体信息处理国家重点实验室);Peking University(北京大学);The Chinese University of Hong Kong(香港中文大学);Wukong Lab, iKingtec(iKingtec悟空实验室);Beijing University of Posts and Telecommunications(北京邮电大学)
Interactive Spatial-Frequency Fusion Mamba for Multi-Modal Image Fusion
School of Future Technology, Dalian University of Technology and the Key Laboratory of Data Science and Smart Education (Hainan Normal University), Ministry of Education(未来技术学院,大连理工大学和数据科学与智能教育关键实验室(海南师范大学),教育部);Affiliated Zhongshan Hospital of Dalian University(大连大学附属中山医院);School of Computer Science and Artificial Intelligence, Wuhan University of Technology(计算机科学与人工智能学院,武汉理工大学);Central Hospital of Dalian University of Technology(大连理工大学中心医院);School of Information and Communication Engineering, Dalian University of Technology(信息与通信工程学院,大连理工大学)
M2IST: Multi-Modal Interactive Side-Tuning for Efficient Referring Expression Comprehension
College of Electronics and Information Engineering, Sichuan University(四川大学电子信息学院);College of Systems Engineering, National University of Defense Technology(国防科技大学系统工程学院);State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);School of Engineering, Westlake University(西湖大学工程学院);College of Computer Science and Cyber Security (Pilot Software College), Chengdu University of Technology(成都理工大学计算机与网络安全学院(软件学院));Yunnan Key Laboratory of Software Engineering, Yunnan University(云南大学云南省软件工程重点实验室)
Interactive Test-Time Adaptation with Reliable Spatial-Temporal Voxels for Multi-Modal Segmentation
Centre for Advanced Robotics Technology Innovation (CARTIN), Nanyang Technological University(南洋理工大学先进机器人技术创新中心);Department of Electrical and Computer Engineering, National University of Singapore(新加坡国立大学电气与计算机工程系);School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院);School of Mechanical and Aerospace Engineering, Nanyang Technological University(南洋理工大学机械与航空航天工程学院)
MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusion
organization= College of Computer Science;Technology, Jilin University , city= Changchun , country= China;organization= Georgia Institute of Technology , city= Atlanta , country= USA;organization= Qingdao Institute of Software, College of Computer Science;Technology, China University of Petroleum (East China) , city= Qingdao , country= China;organization= Institute for Math \& AI, Wuhan University , city= Wuhan , country= China;organization= University of Michigan, Ann Arbor , country= USA;organization= Thrust of Artificial Intelligence, Hong Kong University of Science;organization= School of Artificial Intelligence;Computer Science, Nantong University , city= Nantong , country= China
Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignment
School of Electrical Engineering, KAIST(韩国科学技术院电气工程学院);Department of Electrical Engineering, POSTECH(浦项工科大学电气工程系);School of Engineering and Applied Sciences at Harvard University(哈佛大学工程与应用科学学院)
InterPruner: Interactive Structured Pruning via Taylor-Implicit Criterion and Language-Prior Modulator for Multimodal Object Detection
Beijing University of Technology(北京工业大学);Southwest University(西南大学);China University of Geosciences Wuhan(中国地质大学(武汉));Tsinghua University(清华大学);Beijing Institute of Technology(北京理工大学);Yunnan Normal University(云南师范大学);Beijing Forestry University(北京林业大学);Ghent University(根特大学)
Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editing
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算机科学与技术学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Peng Cheng Laboratory(鹏城实验室)
Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
Imperial College London(伦敦帝国学院);University of Manchester(曼彻斯特大学);HiThink Research(HiThink研究院);Soochow University(苏州大学);Hong Kong Baptist University(香港 Baptist大学);Idiap Research Institute(Idiap研究 institute);Meta AI;Fudan University(复旦大学)
MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning
Chinese University of Hong Kong, Hong Kong SAR, China(香港中文大学);Hunyuan Group, Tencent(腾讯洪音集团);Institute of Automation, the Chinese Academy of Sciences, Beijing, China(中国科学院自动化研究所);Dalian University of Technology, Dalian, China(大连理工大学);Stanford University, Stanford, USA(斯坦福大学)
Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室)