Conversational AI systems are becoming famous in day to day lives. In this paper, we are trying to address the following key question: To identify whether design, as well as development efforts for search oriented conversational AI are successful or not.It is tricky to define 'success' in the case of conversational AI and equally tricky part is to use appropriate metrics for the evaluation of conversational AI. We propose four different perspectives namely user experience, information retrieval, linguistic and artificial intelligence for the evaluation of conversational AI systems. Additionally, background details of conversational AI systems are provided including desirable characteristics of personal assistants, differences between chatbot and an AI based personal assistant. An importance of personalization and how it can be achieved is explained in detail. Current challenges in the development of an ideal conversational AI (personal assistant) are also highlighted along with guidelines for achieving personalized experience for users.
Evaluating the Robustness of Conversational Recommender Systems by Adversarial Examples
Center for Intelligent Information Retrieval（智能信息检索中心）；Manning College of Information and Computer Sciences（曼宁信息与计算机科学学院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Can Conversational Temporal Dynamics Improve Depression Detection in Dyads? A Preliminary Investigation in Multi-Modality Perspectives
Thomas Lord Department of Computer Science, University of Southern California, USA（汤姆斯·劳德计算机科学系，南加州大学，美国）；Signal Analysis and Interpretation Laboratory (SAIL), University of Southern California, USA（信号分析与解释实验室（SAIL），南加州大学，美国）
Evaluating LLMs on Conversational Text-to-SQL under Chain Ambiguity and Intent Drift
The Hong Kong Polytechnic University（香港理工大学）；City University of Macau（澳门城市大学）；Jilin University（吉林大学）；Beihang University（北京航空航天大学）；Jinan University（暨南大学）
How Many Parameters Does it Take to Change a Light Bulb? Evaluating Performance in Self-Play of Conversational Games as a Function of Model Characteristics
University of Potsdam（波茨坦大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）
JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models
Kyoto University（京都大学）；Hohai University（河海大学）；The University of Tokyo（东京大学）；University of Science and Technology of China（中国科学技术大学）；Hong Kong Polytechnic University（香港理工大学）
Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
Technion - Israel Institute of Technology（技术ion-以色列理工学院）；University of Oxford（牛津大学）；University of Zagreb, FER（Zagreb大学，FER）；Kempner Institute, Harvard University（Kempner研究所，哈佛大学）；University of Edinburgh（爱丁堡大学）
Evaluating User Experience in Conversational Recommender Systems: A Systematic Review Across Classical and LLM-Powered Approaches
School of Computer Science, University of Technology Sydney（技术科技大学计算机科学学院）；The Applied College, King Abdulaziz University（国王阿卜杜勒阿齐兹大学应用学院）；Australian Institute of Health Innovation, Macquarie University（麦考瑞大学健康创新研究所）
Factors That Influence the Adoption of AI-enabled Conversational Agents (AICAs) as an Augmenting Therapeutic Tool by Frontline Healthcare Workers: From Technology Acceptance Model 3 (TAM3) Lens -- A Systematic Mapping Review
A Scoping Review of the Ethical Perspectives on Anthropomorphising Large Language Model-Based Conversational Agents
Institute of Biomedical Ethics and History of Medicine, University of Zürich（苏黎世大学生物医学伦理与医学史研究所）；Dalle Molle Institute for Artificial Intelligence (IDSIA), SUPSI（瑞士SUPSI人工智能研究所）；ETH Zürich（苏黎世联邦理工学院）；Institute for Implementation Science in Health Care, University of Zürich（苏黎世大学医疗实施科学研究所）；Department of Management, Technology and Economics, ETH Zürich（苏黎世联邦理工学院管理、技术与经济系）；Dipartimento Tecnologie Innovative, SUPSI（SUPSI创新技术系）；Management in Networked and Digital Societies (MINDS) Department, Kozminski University（科兹明斯基大学网络化与数字化社会管理系）
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tencent（腾讯）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Beihang University（北京航空航天大学）；The Hong Kong University of Science and Technology（香港科技大学）
MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；National Taiwan University（国立台湾大学）；Massachusetts Institute of Technology（麻省理工学院）；Carnegie Mellon University（卡内基梅隆大学）；Meta AI
