Conversational LLMs function as black box systems, leaving users guessing about why they see the output they do. This lack of transparency is potentially problematic, especially given concerns around bias and truthfulness. To address this issue, we present an end-to-end prototype-connecting interpretability techniques with user experience design-that seeks to make chatbots more transparent. We begin by showing evidence that a prominent open-source LLM has a "user model": examining the internal state of the system, we can extract data related to a user's age, gender, educational level, and socioeconomic status. Next, we describe the design of a dashboard that accompanies the chatbot interface, displaying this user model in real time. The dashboard can also be used to control the user model and the system's behavior. Finally, we discuss a study in which users conversed with the instrumented system. Our results suggest that users appreciate seeing internal states, which helped them expose biased behavior and increased their sense of control. Participants also made valuable suggestions that point to future directions for both design and machine learning research. The project page and video demo of our TalkTuner system are available at https://bit.ly/talktuner-project-page
Leveraging Large Language Models for Patient Engagement: The Power of Conversational AI in Digital Health
IBM T. J. Watson Research Center(IBM TJ沃森研究中心);Morehouse School of Medicine(莫尔豪斯医学院);Cleveland Clinic Foundation(克利夫兰诊所基金会);Queen’s University(女王大学);Red Hat(红帽公司)
Towards Ontology-Based Descriptions of Conversations with Qualitatively-Defined Concepts
Loria, CNRS, Université de Lorraine, Nancy, France(Loria研究所、法国国家科学研究中心、洛林大学、南希分校);Université Sorbonne Paris Nord, CNRS, Laboratoire d'Informatique de Paris Nord, LIPN, F-93430 Villetaneuse, France(巴黎北大学、法国国家科学研究中心、巴黎北信息技术实验室、LIPN、法国)
Towards Explainable Conversational AI for Early Diagnosis with Large Language Models
Department of Information and Communication Technology(信息与通信技术系);Bangladesh University of Professionals(孟加拉专业大学);Institute of Information Technology(信息技术研究所);Jahangirnagar University(贾汗吉尔纳瓦德大学)
Open-Source Conversational AI with SpeechBrain 1.0
Concordia University(康考迪亚大学);Mila-Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Samsung AI Center Cambridge(三星剑桥人工智能中心);University of Cambridge(剑桥大学);Avignon University(阿维尼翁大学);Laval University(拉瓦尔大学);Zaion(扎伊翁);Fondazione Bruno Kessler(布鲁诺·凯塞勒基金会);Aalto University(阿尔托大学);University of Bologna(博洛尼亚大学);Telecom Paris(巴黎电信学院);University of Edinburgh(爱丁堡大学);Inria(法国国家信息与自动化研究所);Silo AI(西罗人工智能公司);Idiap(伊迪亚普研究所);KU Leuven(鲁汶大学);EPFL(洛桑联邦理工学院);University of Trento(特伦托大学);National Institute of Informatics - Tokyo(东京情报学研究所)
RECOVER: Designing a Large Language Model-based Remote Patient Monitoring System for Postoperative Gastrointestinal Cancer Care
University of California, Irvine(加州大学尔湾分校);Northeastern University(东北大学);Johns Hopkins University(约翰霍普金斯大学);New York University(纽约大学);MedStar Health Research Institute(梅斯塔健康研究院);Creighton University(克里普顿大学);Georgetown University(乔治城大学)
UniAutoML: A Human-Centered Framework for Unified Discriminative and Generative AutoML with Large Language Models
Glasgow College, University of Electronic Science and Technology of China(电子科技大学格拉斯哥学院);School of Engineering, University of Glasgow(格拉斯哥大学工程学院);Toursun Synbio(同舜合成生物);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院);School of Information and Software Engineering, University of Electronic Science and Technology Of China(电子科技大学信息与软件工程学院);Research Institute, Ubtech Robotics Corp(优必选机器人公司研究院);Qiyuan Lab(启源实验室);Department of Computer Science, Johns Hopkins University(约翰斯·霍普金斯大学计算机科学系)