当RAG聊天机器人暴露其后端：对患者面向医疗AI隐私和安全风险的匿名案例研究
When RAG Chatbots Expose Their Backend: An Anonymized Case Study of Privacy and Security Risks in Patient-Facing Medical AI
AI总结：
研究通过匿名案例分析，揭示了患者面向RAG聊天机器人中隐私和安全风险，指出通过浏览器工具可发现关键漏洞，强调部署前需独立审查。
摘要：
背景：面向患者医疗聊天机器人基于检索增强生成（RAG）技术日益推广，以提供可及的健康信息。AI辅助开发降低了构建门槛，但仍需严格的安全、隐私和治理控制。目标：报告一个公开可访问的患者面向医疗RAG聊天机器人的匿名非破坏性安全评估，并识别生成AI在健康领域安全部署的治理教训。方法：我们采用两阶段策略。首先，Claude Opus 4.6支持基于提示的探索性测试和结构化漏洞假设。其次，候选发现通过Chrome开发者工具手动验证，检查浏览器可见的网络流量、负载、API架构、配置对象和存储的交互数据。结果：LLM辅助阶段识别出关键漏洞：敏感系统和RAG配置通过客户端-服务器通信暴露，而非限制在服务器端。手动验证确认普通浏览器检查可收集系统提示、模型和嵌入配置、检索参数、后端端点、API架构、文档和分块元数据、知识库内容以及最近1000次患者-聊天机器人对话。部署也与其隐私保证相矛盾：完整对话记录，包括与健康相关的查询，可在无需身份验证的情况下检索。结论：患者面向RAG聊天机器人中存在严重的隐私和安全故障，可通过标准浏览器工具发现，无需专业知识或身份验证；独立审查应成为部署的前提。商业LLM加速了这一评估，包括在假开发人员身份下；为审计员提供的帮助同样可供对手使用。
英文摘要：
Background: Patient-facing medical chatbots based on retrieval-augmented generation (RAG) are increasingly promoted to deliver accessible, grounded health information. AI-assisted development lowers the barrier to building them, but they still demand rigorous security, privacy, and governance controls. Objective: To report an anonymized, non-destructive security assessment of a publicly accessible patient-facing medical RAG chatbot and identify governance lessons for safe deployment of generative AI in health. Methods: We used a two-stage strategy. First, Claude Opus 4.6 supported exploratory prompt-based testing and structured vulnerability hypotheses. Second, candidate findings were manually verified using Chrome Developer Tools, inspecting browser-visible network traffic, payloads, API schemas, configuration objects, and stored interaction data. Results: The LLM-assisted phase identified a critical vulnerability: sensitive system and RAG configuration appeared exposed through client-server communication rather than restricted server-side. Manual verification confirmed that ordinary browser inspection allowed collection of the system prompt, model and embedding configuration, retrieval parameters, backend endpoints, API schema, document and chunk metadata, knowledge-base content, and the 1,000 most recent patient-chatbot conversations. The deployment also contradicted its privacy assurances: full conversation records, including health-related queries, were retrievable without authentication. Conclusions: Serious privacy and security failures in patient-facing RAG chatbots can be identified with standard browser tools, without specialist skills or authentication; independent review should be a prerequisite for deployment. Commercial LLMs accelerated this assessment, including under a false developer persona; assistance available to auditors is equally available to adversaries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
