As LLM-powered scams proliferate, black-box forensics for conversational LLM agents offers a path to accountability for systems hidden behind anonymous endpoints. Identifying the base model behind a chatbot endpoint (attribution), without model parameter access or knowledge of the hidden system prompt, would let investigators trace AI-enabled scams back to the providers whose models power them. Detecting when two endpoints run the exact same system prompt (fingerprinting), even one novel and unseen, would link individual scams into criminal networks and expose silent API changes. We conduct an empirical investigation of both capabilities. Our attribution classifiers identify the base model behind an agent with 98% accuracy from a few turns of non-adversarial conversation. Attribution of system prompts, while possible, requires retraining on a large amount of data for each prompt; system prompts in the wild are unbounded and ever-changing, making this approach costly. To tackle this more open-ended setting, our cross-encoder fingerprinting method achieves an AUC of 0.768 and an F1 of 0.703 on entirely unseen system prompts, and aggregating 50 interaction conversations from each target agent boosts AUC to 0.943. Conversational agents with unseen system prompts can thus be fingerprinted with robust accuracy from a few turns of ordinary conversation.
Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks
Pine AI;The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Duke University(杜克大学);University of Michigan(密歇根大学);Stevens Institute of Technology(史蒂文斯理工学院);Johns Hopkins University(约翰霍普金斯大学);University of Notre Dame(圣母大学);The Pennsylvania State University(宾夕法尼亚州立大学)
AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation
School of Computer Science and Engineering, Key Laboratory of Computer Network and Information Integration, Ministry of Education, Southeast University(计算机科学与工程学院、计算机网络与信息集成重点实验室、教育部、东南大学)
Automated Evaluation of Multi-Turn Dialogues in In-Car Conversational Assistants
Friedrich-Alexander-Universität Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学);BMW Group(宝马集团);Technical University of Munich(慕尼黑工业大学);RWTH Aachen University(亚琛工业大学);fortiss GmbH(fortiss有限公司)
daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);University of Chicago(芝加哥大学);The Chinese University of Hong Kong(香港中文大学);East China Normal University(华东师范大学)
Psychologically-Grounded Graph Modeling for Interpretable Depression Detection
Singapore Institute of Technology(新加坡科技学院);Nanyang Technological University(南洋理工大学);ELLIS Institute Finland(芬兰ELLIS研究所);University of Turku(图尔库大学);Princess Nourah bint Abdulrahman University(普里西拉努拉·本·阿卜杜勒拉赫曼大学)
Performance of a large language model-Artificial Intelligence based chatbot for counseling patients with sexually transmitted infections and genital diseases
University College of Medical Sciences and Guru Teg Bahadur Hospital(大学医学院与古鲁·特格·巴哈杜尔医院);HeHealth Inc.(HeHealth公司)