Current LLM assistants are powerful at answering questions, but they have limited access to the behavioral context that reveals when and where a user is struggling. We present a gaze-grounded multimodal LLM assistant that uses egocentric video with gaze overlays to identify likely points of difficulty and target follow-up retrospective assistance. We instantiate this vision in a controlled study (n=36) comparing the gaze-aware AI assistant to a text-only LLM assistant. Compared to a conventional LLM assistant, the gaze-aware assistant was rated as significantly more accurate and personalized in its assessments of users' reading behavior and significantly improved people's ability to recall information. Users spoke significantly fewer words with the gaze-aware assistant, indicating more efficient interactions. Qualitative results underscored both perceived benefits in comprehension and challenges when interpretations of gaze behaviors were inaccurate. Our findings suggest that gaze-aware LLM assistants can reason about cognitive needs to improve cognitive outcomes of users.
SmartBench: Evaluating LLMs in Smart Homes with Anomalous Device States and Behavioral Contexts
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Jilin University（吉林大学）；The Chinese University of Hong Kong（香港中文大学）；Pengcheng Laboratory（鹏城实验室）
RetinalGPT: A Retinal Clinical Preference Conversational Assistant Powered by Large Vision-Language Models
School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学计算与增强智能学院）；School of Computing, Clemson University（克莱姆森大学计算机学院）；Washington University School of Medicine in St. Louis（圣路易斯华盛顿大学医学院）；The Keck School of Medicine of the University of Southern California（南加州大学凯克医学院）；Mayo Clinic（梅奥诊所）；Banner Alzheimer’s Institute（班纳阿尔茨海默病研究所）
Better Adherence, Richer Context: A Field Evaluation of LLM-Powered Conversational Voice Diaries for Sleep
The Johns Hopkins University（约翰霍普金斯大学）；Department of Psychiatry and Behavioral Sciences, The Johns Hopkins University School of Medicine（精神病学与行为科学系，约翰霍普金斯大学医学院）
IndEgo: A Dataset of Industrial Scenarios and Collaborative Work for Egocentric Assistants
Fraunhofer IPK（弗劳恩霍夫研究所）；Technical University of Berlin（柏林技术大学）；University of Tübingen（图宾根大学）；RWTH Aachen University（亚琛工业大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）
Multimodal Large Language Models for Bioimage Analysis
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室）；Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V.（莱布尼茨分析科学研究所）
Gaze-supported Large Language Model Framework for Bi-directional Human-Robot Interaction
Technical University of Munich（慕尼黑技术大学）；Robert Bosch GmbH, Corporate Research（博世集团，企业研究）；Centre for Applied Autonomous Sensor Systems (AASS)（应用自主传感器系统中心（AASS））
Beyond the Personal Assistant: How Expectations for Enterprise AI in Teamwork Diverged as Generative AI Took Shape, 2023-2025
Carnegie Mellon University（卡内基梅隆大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Pennsylvania（宾夕法尼亚大学）；Stanford University（斯坦福大学）；Emory University（埃默里大学）
Autonomous Artificial Intelligence Agents for Clinical Decision Making in Oncology
Heidelberg University Hospital（海德堡大学医院）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；Else Kroener Fresenius Center for Digital Health（Else Kroener Fresenius数字健康中心）；Technical University Dresden（德累斯顿工业大学）；University of St Andrews（圣安德鲁斯大学）；Heidelberg University（海德堡大学）；University Hospital RWTH Aachen（亚琛工业大学附属医院）；University Hospital Dresden（德累斯顿大学医院）；University Medical Center Mainz（美因茨大学医学中心）；Technical University of Munich (TUM)（慕尼黑工业大学）；Center for Personalized Medicine (ZPM)（个性化医学中心）；German Cancer Consortium (DKTK)（德国癌症联盟）；Google Cloud（谷歌云）；Institute for Cancer Research（癌症研究所）；Royal Marsden Hospital（皇家马斯登医院）
University of Maryland, College Park（马里兰大学帕克分校）；Microsoft Research（微软研究院）
Mobile agents that can leverage help from humans can potentially accomplish more complex tasks than they could entirely on their own. We develop "Help, Anna!" (HANNA), an interactive photo-realistic simulator in which an agent fulfills object-finding tasks by requesting and interpreting natural language-and-vision assistance. An agent solving tasks in a HANNA environment can leverage simulated human assistants, called ANNA (Automatic Natural Navigation Assistants), which, upon request, provide natural language and visual instructions to direct the agent towards the goals. To address the HANNA …
