VelLMes: A high-interaction AI-based deception framework
Czech Technical University in Prague(捷克技术大学布拉格)
Muris Sladić, Veronica Valeros, Carlos Catania, Sebastian Garcia
备注:
9 pages. 9 figures. 1 table. This is a preprint of a paper that was presented at the Active Defense and Deception Workshop colocated with IEEE EuroS&P 2025 conference
目前,基于大型语言模型(LLM)的最先进欺骗系统非常少。现有系统仅限于模拟单一类型的服务,主要是SSH shell。这些系统——以及不基于LLM的欺骗技术——缺乏包含人类攻击者的广泛评估。生成式AI最近已成为网络安全研究人员和从业者的宝贵资产,网络欺骗领域也不例外。研究人员已经展示了如何利用LLM创建逼真的蜜标(honeytokens)、虚假用户,甚至可用作蜜罐的模拟系统。本文提出了一种名为VelLMes的基于AI的欺骗框架,该框架可以模拟多种协议和服务,如SSH Linux shell、MySQL、POP3和HTTP。所有这些都可以部署并用作蜜罐,因此VelLMes根据用户需求提供了多种欺骗设计选择。VelLMes的设计目标是接受人类攻击,因此交互性和真实感对其性能至关重要。我们评估了生成能力和欺骗能力。生成能力通过针对LLM的单元测试进行评估。单元测试结果表明,通过精心设计的提示(prompting),LLM可以生成逼真的响应,部分LLM的通过率达到100%。在SSH Linux shell的情况下,我们使用89名人类攻击者评估了欺骗能力。结果显示,约有30%的攻击者认为,当他们被分配到一个基于LLM的蜜罐时,他们正在与真实系统交互。最后,我们在互联网上部署了10个SSH Linux shell蜜罐实例,以捕获真实世界的攻击。对这些攻击的分析表明,模拟Linux shell的LLM蜜罐能够很好地应对互联网上的非结构化和意外攻击,对大多数发出的命令都能正确响应。
英文摘要:
There are very few SotA deception systems based on Large Language Models. The existing ones are limited only to simulating one type of service, mainly SSH shells. These systems - but also the deception technologies not based on LLMs - lack an extensive evaluation that includes human attackers. Generative AI has recently become a valuable asset for cybersecurity researchers and practitioners, and the field of cyber-deception is no exception. Researchers have demonstrated how LLMs can be leveraged to create realistic-looking honeytokens, fake users, and even simulated systems that can be used as honeypots. This paper presents an AI-based deception framework called VelLMes, which can simulate multiple protocols and services such as SSH Linux shell, MySQL, POP3, and HTTP. All of these can be deployed and used as honeypots, thus VelLMes offers a variety of choices for deception design based on the users' needs. VelLMes is designed to be attacked by humans, so interactivity and realism are key for its performance. We evaluate the generative capabilities and the deception capabilities. Generative capabilities were evaluated using unit tests for LLMs. The results of the unit tests show that, with careful prompting, LLMs can produce realistic-looking responses, with some LLMs having a 100% passing rate. In the case of the SSH Linux shell, we evaluated deception capabilities with 89 human attackers. The results showed that about 30% of the attackers thought that they were interacting with a real system when they were assigned an LLM-based honeypot. Lastly, we deployed 10 instances of the SSH Linux shell honeypot on the Internet to capture real-life attacks. Analysis of these attacks showed us that LLM honeypots simulating Linux shells can perform well against unstructured and unexpected attacks on the Internet, responding correctly to most of the issued commands.
DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);College of AI, Tsinghua University(清华大学人工智能学院);Shanghai Qi Zhi Institute(上海启智研究所);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室)
Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL
Department of Computer Science, University of Maryland(大学计算机科学系);Northwestern University(西北大学);School of Computer Science, University of Sydney(悉尼大学计算机科学学院);Information Sciences Institute, University of Southern California(南加州大学信息科学研究所)
HoneyGPT: Breaking the Trilemma in Terminal Honeypots with Large Language Model
Institute of Information Engineering CAS(中国科学院信息工程研究所);School of Cyber Security UCAS(中国科学院大学网络空间安全学院);Sangfor Technologies Inc.(深信服科技股份有限公司);Virginia Tech(弗吉尼亚理工大学);Shenzhen Institute of Advanced Technology CAS(中国科学院深圳先进技术研究院)
LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactions
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Fudan University(复旦大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学)
CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulation
University of Copenhagen(哥本哈根大学);IIIT Ranchi(印度信息技术学院兰契分校);ISI Kolkata(印度统计学院加尔各答分校);NIT Andhra Pradesh(印度国家理工学院安得拉邦分校);IGDTUW(英迪拉·甘地德里技术女子大学);IIT Kharagpur(印度理工学院卡哈拉格普尔分校);Google DeepMind(谷歌DeepMind);Google(谷歌);AI Institute, University of South Carolina(南卡罗来纳大学人工智能研究所)
The Stepwise Deception: Simulating the Evolution from True News to Fake News with LLM Agents
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学北京校区人工智能学院);MBZUAI;Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MoE(下一代智能搜索与推荐工程研究中心,教育部);School of Artifcial Intelligence, Wuhan University(武汉大学人工智能学院)
RogueAI: A Reverse Turing Test for Detecting Licensed AI Deception in Dialogue
AILab, MIGe, University of Trieste(的里雅斯特大学);Computational Statistics and Machine Learning, Istituto Italiano di Tecnologia(意大利理工学院);DIA, University of Trieste(的里雅斯特大学)
Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislation
Centre for Responsible AI, IIT Madras(IIT马德拉斯责任人工智能中心);Indian Institute of Technology Madras(印度马德拉斯理工学院);Princeton University(普林斯顿大学);Stanford University(斯坦福大学)
QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents
McGill University(麦吉尔大学);Mila - Quebec AI Institute(魁北克人工智能研究所);University of Cambridge(剑桥大学);MBZUAI - Mohamed bin Zayed University of Artificial Intelligence(MBZUAI - 摩苏尔·本·扎耶德人工智能大学);University of Toronto(多伦多大学);Salesforce
AdvancedShelLM: A Stateful Multi-Agent LLM Honeypot for SSH Deception
Faculty of Electrical Engineering, Czech Technical University in Prague(布拉格捷克技术大学电气工程学院);School of Engineering, Universidad Nacional de Cuyo(门多萨国立大学工程学院)
InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles
Nankai University(南开大学);Shanghai Innovation Institute(上海创新研究院);Shanghai AI Laboratory(上海人工智能实验室);Fudan University(复旦大学);Johns Hopkins University(约翰霍普金斯大学);Wuhan University(武汉大学);University of Science and Technology of China(中国科学技术大学)
A Scoping Review of the Ethical Perspectives on Anthropomorphising Large Language Model-Based Conversational Agents
Institute of Biomedical Ethics and History of Medicine, University of Zürich(苏黎世大学生物医学伦理与医学史研究所);Dalle Molle Institute for Artificial Intelligence (IDSIA), SUPSI(瑞士SUPSI人工智能研究所);ETH Zürich(苏黎世联邦理工学院);Institute for Implementation Science in Health Care, University of Zürich(苏黎世大学医疗实施科学研究所);Department of Management, Technology and Economics, ETH Zürich(苏黎世联邦理工学院管理、技术与经济系);Dipartimento Tecnologie Innovative, SUPSI(SUPSI创新技术系);Management in Networked and Digital Societies (MINDS) Department, Kozminski University(科兹明斯基大学网络化与数字化社会管理系)
AIIPot: Adaptive Intelligent-Interaction Honeypot for IoT Devices
University of Dschang(德昌大学);University of Southampton(南安普顿大学);Air Force Research Laboratory(空军研究实验室);Combat Capabilities Development Command Army Research Laboratory(作战能力发展司令部陆军研究实验室)
Why did My Robot Just Change Personality? Prompting Guidelines for a Grounded Robot Persona in LLM-Based HRI
RPTU Kaiserslautern(凯泽斯劳滕工业大学);Linköping University(林雪平大学);University of Hertfordshire(赫特福德大学);Eindhoven University of Technology(埃因霍温理工大学);Australian National University(澳大利亚国立大学);University of Cambridge(剑桥大学);University of Naples Federico II(那不勒斯费德里科二世大学);Ben-Gurion University of the Negev(本-古里安大学内盖夫分校);The Azrieli National Center for Autism and Neurodevelopment Research(阿兹列利国家自闭症与神经发育研究中心)
UT Austin — Department of Computer Science(德克萨斯大学奥斯汀分校计算机科学系);UT Austin — Department of Communication Studies(德克萨斯大学奥斯汀分校传播学系);UT Austin — Technology & Information Policy Institute(德克萨斯大学奥斯汀分校科技与信息政策研究所);UT Austin — Department of Psychology(德克萨斯大学奥斯汀分校心理学系);UT School of Law(德克萨斯大学法学院);UT Austin — Department of English(德克萨斯大学奥斯汀分校英语系);El Colegio Mexiquense, A.C. — Seminario de Instituciones, Sociedad Civil y Políticas Públicas(墨西哥埃尔科尔西奥学院,A.C. — 社会与公共政策研讨会);Sony AI(索尼人工智能)