In contrast to classical cognitive science which studied brains in isolation, ecological approaches focused on the role of the body and environment in shaping cognition. Similarly, in this thesis we adopt an ecological approach to grounded natural language understanding (NLU) research. Grounded language understanding studies language understanding systems situated in the context of events, actions and precepts in naturalistic/simulated virtual environments. Where classic research tends to focus on designing new models and optimization methods while treating environments as given, we explore the potential of environment design for improving data collection and model development. We developed novel training and annotation approaches for procedural text understanding based on text-based game environments. We also drew upon embodied cognitive linguistics literature to propose a roadmap for grounded NLP research, and to inform the development of a new benchmark for measuring the progress of large language models on challenging commonsense reasoning tasks. We leveraged the richer supervision provided by text-based game environments to develop Breakpoint Transformers, a novel approach to modeling intermediate semantic information in long narrative or procedural texts. Finally, we integrated theories on the role of environments in collective human intelligence to propose a design for AI-augmented "social thinking environments" for knowledge workers like scientists.
An Ethically Grounded LLM-Based Approach to Insider Threat Synthesis and Detection
Beacom College of Computer \& Cyber Sciences Dakota State University Madison, SD, USA；College of Arts \& Sciences Dakota State University Madison, SD, USA
XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models
Imperial College London（帝国理工学院）；Technical University of Munich（慕尼黑工业大学）；University of Southampton（南安普顿大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Johns Hopkins University（约翰霍普金斯大学）
Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective
City University of Hong Kong（香港城市大学）；Northwestern University（西北大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Oxford University（牛津大学）；Allen Institute for AI (AI2)（人工智能研究所）；University of Washington（华盛顿大学）；National University of Singapore（新加坡国立大学）；The Hong Kong Polytechnic University（香港理工大学）
Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures Inside Arguments
Institute of Artificial Intelligence, School of Computer Science and Technology（计算机科学与技术学院人工智能研究所）；Soochow University（苏州大学）；Huawei Cloud（华为云）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）
IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provinces
MBZUAI（MBZUAI（穆罕默德·本·扎耶德人工智能大学））；Universitas Indonesia（印度尼西亚大学）；Royal Melbourne Institute of Technology（皇家墨尔本理工大学）；Quantic School of Business and Technology（匡蒂克商学院）；The University of Melbourne（墨尔本大学）
