CG-Probes：从患者查询嵌入中恢复护栏方向
CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings
浏览论文内容
中文总结 AI 辅助
本研究提出临床护栏探针（CG-Probes），通过均值差法从患者查询嵌入中恢复医疗紧迫性等风险方向，以低延迟实现与开放权重LLM相当的风险评估，助力临床护栏设置。
中文摘要 AI 辅助
面向患者的AI助手有望为患者提供宝贵的支持，但传入的查询可能带来医疗风险。为了创建护栏，我们与肿瘤学家合作定义了三个序数风险轴：医疗紧迫性、心理紧迫性和主题敏感性。我们提出了临床护栏探针（CG-Probes）来测量查询嵌入中的风险。我们通过均值差方法在冻结嵌入器的归一化嵌入空间中为每个轴进行探测，将每个轴视为潜在的线性方向。为了训练探针，我们使用BERTopic对79,658个捷克肿瘤学搜索查询进行聚类，并利用这些聚类通过少样本提示生成具有对比风险级别的查询对。我们在200个查询（90个真实，110个合成）上评估了该方法，每个查询由两位肿瘤学家评分，并与两个开放权重LLM和一个前沿LLM进行比较。我们发现基于紧迫性的轴可以作为线性方向恢复，并且探针在延迟的一小部分内与开放权重LLM竞争（在二次加权kappa上无显著差异）。每个轴产生一个标量分数，临床医生可以检查并使用该分数来设置升级阈值。该流程仅需要搜索日志、轴定义和嵌入模型的黑盒访问，表明其可跨医疗保健领域转移。对新查询和轴的稳健验证仍是未来的工作。
英文摘要
Patient-facing AI assistants promise valuable support to patients, but incoming queries can pose medical risks. To create guardrails, we work with oncologists to define three ordinal risk axes: Medical Urgency, Psychological Urgency, and Topic Sensitivity. We propose Clinical Guardrail Probes (CG-Probes) to measure the risks from query embeddings. We probe for each axis in the normalized embedding space of frozen embedders via the difference-in-means method, treating each axis as a potential linear direction. To train the probes, we cluster 79,658 Czech oncology search queries with BERTopic and use these clusters to generate pairs of queries with contrastive risk levels via few-shot prompting. We evaluate the approach on 200 queries (90 real, 110 synthetic), each graded by two oncologists, against two open-weight LLMs and a frontier LLM. We find that urgency-based axes are recoverable as linear directions, and the probes are competitive with open-weight LLMs (no significant differences in quadratic-weighted kappa) at a fraction of the latency. Each axis yields a scalar score that clinicians can inspect and use to set escalation thresholds. The pipeline requires only search logs, axis definitions, and black-box access to the embedding model, suggesting transferability across healthcare domains. Robust validation on new queries and axes remains future work.
发表机构
- Masaryk University（马萨里克大学）
- University Hospital Brno（布尔诺大学医院）
- Masaryk Memorial Cancer Institute（马萨里克纪念癌症研究所）
机构由 AI 辅助整理，请以论文原文为准。
