In recent years, the rapid development of AI systems has brought about the benefits of intelligent services but also concerns about security and reliability. By fostering appropriate user reliance on an AI system, both complementary team performance and reduced human workload can be achieved. Previous empirical studies have extensively analyzed the impact of factors ranging from task, system, and human behavior on user trust and appropriate reliance in the context of one-step decision making. However, user reliance on AI systems in tasks with complex semantics that require multi-step workflows remains under-explored. Inspired by recent work on task decomposition with large language models, we propose to investigate the impact of a novel Multi-Step Transparent (MST) decision workflow on user reliance behaviors. We conducted an empirical study (N = 233) of AI-assisted decision making in composite fact-checking tasks (i.e., fact-checking tasks that entail multiple sub-fact verification steps). Our findings demonstrate that human-AI collaboration with an MST decision workflow can outperform one-step collaboration in specific contexts (e.g., when advice from an AI system is misleading). Further analysis of the appropriate reliance at fine-grained levels indicates that an MST decision workflow can be effective when users demonstrate a relatively high consideration of the intermediate steps. Our work highlights that there is no one-size-fits-all decision workflow that can help obtain optimal human-AI collaboration. Our insights help deepen the understanding of the role of decision workflows in facilitating appropriate reliance. We synthesize important implications for designing effective means to facilitate appropriate reliance on AI systems in composite tasks, positioning opportunities for the human-centered AI and broader HCI communities.
Integrating Transparent Models, LLMs, and Practitioner-in-the-Loop: A Case of Nonprofit Program Evaluation
LBJ School of Public Affairs, The University of Texas at Austin(德克萨斯大学奥斯汀分校劳伦斯·伯克曼公共事务学院);Gradel Institute of Charity, University of Oxford(牛津大学格拉德利慈善研究所);Michael & Susan Dell Foundation(迈克尔与苏珊·德尔基金会)
Text-to-Layout: A Generative Workflow for Drafting Architectural Floor Plans Using LLMs
Department of Civil and Environmental Engineering, University of Houston(土木与环境工程系,休斯顿大学);Center for Transportation Research, The University of Texas at Austin(交通研究中心,德克萨斯大学奥斯汀分校);Department of Information Science Technology, University of Houston(信息科学与技术系,休斯顿大学)
Learning to Be A Doctor: Searching for Effective Medical Agent Architectures
AGI Lab, Westlake University(西拉雅大学AGI实验室);Henan University(河南大学);Affiliated Hospital of Xuzhou Medical University(徐州医科大学附属医院);Xuzhou Medical University(徐州医科大学);Nanyang Technological University(南洋理工大学);IHPC, Agency for Science, Technology and Research, Singapore(新加坡科技研究局IHPC);CFAR, Agency for Science, Technology and Research, Singapore(新加坡科技研究局CFAR)
Expert-Annotated Embryo Image Dataset with Natural Language Descriptions for Evidence-Based Patient Communication in IVF
Software Competence Center Hagenberg(海因斯贝格软件能力中心);Kepler Universitätsklinikum, Kinderwunsch Zentrum(凯普勒大学医院,生育中心);Wunschkind Klinik Dr. Brunbauer(布鲁纳auer生育诊所)
AeroSpectra Sentinel: An Auditable LLM Prompt-Chaining Decision-Support Workflow for Acute Asthma Risk Assessment from Respiratory Sounds and Clinical Signals
School of Information, Computer, and Communication Technology(信息、计算机与通信技术学院);Sirindhorn International Institute of Technology, Thammasat University(泰国朱拉隆梭国际技术学院)
Vibe Coding in Product Teams: Reconfiguring AI-Assisted Workflows, Prototyping, and Collaboration
Netherlands and MIT Media Lab(荷兰和MIT媒体实验室);Notion Labs(Notion实验室);Collov AI;Cornell University(康奈尔大学);Goizueta Business School, Emory University(埃默里大学戈伊兹埃塔商学院);Centrum Wiskunde & Informatica, The Netherlands and Utrecht University, The Netherlands(荷兰代尔夫特理工大学和乌得勒支大学)
BioProVLA-Agent: An Affordable, Protocol-Driven, Vision-Enhanced VLA-Enabled Embodied Multi-Agent System with Closed-Loop-Capable Reasoning for Biological Laboratory Manipulation
Key Laboratory of Smart Manufacturing in Energy Chemical Process Ministry of Education, East China University of Science and Technology, Shanghai, CN(能源化工过程智能制造教育部重点实验室,东华大学,上海,中国);Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, CN(东华大学计算机科学与工程系,上海,中国);Department of Laboratory Medicine, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, CN(复旦大学附属瑞金医院检验医学科,上海,中国);School of Information Science and Technology, Shihezi University, Shihezi, CN(石河子大学信息科学与技术学院,石河子,中国)
SkinGPT-X: A Self-Evolving Collaborative Multi-Agent System for Transparent and Trustworthy Dermatological Diagnosis
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Department of Dermatology, Tianjin Institute of Integrative Dermatology, Tianjin Academy of Traditional Chinese Medicine Affiliated Hospital(天津中医药大学附属医院天津市中西医结合皮肤病研究所皮肤科);Department of Dermatology, Beijing AnZhen Hospital, Capital Medical University(首都医科大学附属北京安贞医院皮肤科)
Agentic AI Enhances Physician Trust in Clinical Decision Making
Department of Computer Science and Engineering, Lehigh University(里海大学计算机科学与工程系);Department of Epidemiology, Harvard T.H. Chan School of Public Health(哈佛大学陈曾熙公共卫生学院流行病学系);Department of Medicine, Division of Cardiology, University of Florida(佛罗里达大学医学院心脏病学系);McGovern Medical School, University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心麦戈文医学院);Department of Radiology, Massachusetts General Hospital and Harvard Medical School(麻省总医院和哈佛医学院放射学系);McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心麦克威廉姆斯生物医学信息学院);Department of Health Outcomes & Biomedical Informatics, College of Medicine, University of Florida(佛罗里达大学医学院健康结果与生物医学信息学系)