Simulating learner actions helps stress-test open-ended interactive learning environments and prototype new adaptations before deployment. While recent studies show the promise of using large language models (LLMs) for simulating human behavior, such approaches have not gone beyond rudimentary proof-of-concept stages due to key limitations. First, LLMs are highly sensitive to minor prompt variations, raising doubts about their ability to generalize to new scenarios without extensive prompt engineering. Moreover, apparently successful outcomes can often be unreliable, either because domain experts unintentionally guide LLMs to produce expected results, leading to self-fulfilling prophecies; or because the LLM has encountered highly similar scenarios in its training data, meaning that models may not be simulating behavior so much as regurgitating memorized content. To address these challenges, we propose Hyp-Mix, a simulation authoring framework that allows experts to develop and evaluate simulations by combining testable hypotheses about learner behavior. Testing this framework in a physics learning environment, we found that GPT-4 Turbo maintains calibrated behavior even as the underlying learner model changes, providing the first evidence that LLMs can be used to simulate realistic behaviors in open-ended interactive learning environments, a necessary prerequisite for useful LLM behavioral simulation.
BIMgent: Towards Autonomous Building Modeling via Computer-use Agents
Chair of Computing in Civil and Building Engineering, Technical University of Munich, Germany（土木与建筑工程计算系，慕尼黑技术大学）；TUM Georg Nemetschek Institute, Munich, Germany（慕尼黑技术大学Georg Nemetschek研究所）
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Transvascular Implantation Devices and TIDRI（血管植入设备国家重点实验室和TIDRI）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江医学影像人工智能重点实验室）；School of Data Science of Engineering, East China Normal University（华东师范大学工程数据科学学院）；Second Affiliated Hospital and Liangzhu Laboratory, Zhejiang University School of Medicine（浙江大学医学院第二附属医院和良渚实验室）；Alibaba Group（阿里巴巴集团）
SPADE: Self-Play in Adaptive Synthetic Executable Environments
University of Washington（华盛顿大学）；Stanford University（斯坦福大学）；Northeastern University（东北大学）；Carnegie Mellon University（卡内基梅隆大学）；Massachusetts Institute of Technology（麻省理工学院）；National University of Singapore（新加坡国立大学）；Seoul National University（首尔大学）；Stevens Institute of Technology（史蒂文斯理工学院）；University of Chicago（芝加哥大学）
Heterogeneous Adversarial Play in Interactive Environments
Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）；School of Psychological and Cognitive Sciences, Peking University（心理与认知科学学院，北京大学）；School of Computer Science & Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）；State Key Lab of General AI, Peking University（通用人工智能国家重点实验室，北京大学）；Beijing Key Laboratory of Behavior and Mental Health, Peking University（北京行为与心理健康重点实验室，北京大学）；Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing（北京理工大学长江三角洲地区研究院，嘉兴）；Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence（具身智能实验室，北京大学-武汉人工智能研究院）
