arXiv 2602.088892026-02cs.AI

可扩展的德尔菲：用于结构化风险估计的大型语言模型

Scalable Delphi: Large Language Models for Structured Risk Estimation

Tobias Lorenz, Mario Fritz

AI总结：

本文提出可扩展的德尔菲方法，利用大型语言模型进行结构化风险评估，通过多样化的专家人设和迭代优化，实现了高效且准确的风险估计。

摘要：

在高风险领域中，定量风险评估依赖于结构化专家征询来估计不可观测的属性。金标准——德尔菲方法——会产生校准且可审计的判断，但需要数月的协调和专门时间，使严格的风险评估对大多数应用来说难以达到。我们探讨了大型语言模型（LLM）是否可以作为结构化专家征询的可扩展代理。我们提出了可扩展的德尔菲，将经典协议适应到LLM上，具有多样化的专家人设、迭代优化和理由分享。由于目标量通常不可观测，我们开发了一个基于必要条件的评估框架：校准与可验证的代理、对证据的敏感性和与人类专家判断的一致性。我们在AI增强型网络安全风险领域进行评估，使用三个能力基准和独立的人类征询研究。LLM小组在基准真实值上表现出强相关性（皮尔逊r=0.87-0.95），随着证据的增加系统性地改进，并与人类专家小组一致——在一次比较中，比两个人类小组彼此更接近。这表明基于LLM的征询可以将结构化专家判断扩展到传统方法不可行的领域，将征询时间从数月减少到分钟。

英文摘要：

Quantitative risk assessment relies on structured expert elicitation to estimate unobservable properties. The Delphi method produces calibrated, auditable estimates but requires months of coordination and specialist time, placing rigorous risk assessment out of reach for most applications. We propose Scalable Delphi, adapting the classical protocol for LLMs with diverse expert personas, iterative refinement, and rationale sharing. Beyond lowering cost, this makes the assessment analyzable and dynamic. Rationales and revision histories record what each estimate rests on, information can be ablated to test which evidence matters, and the elicitation can be rerun with new evidence, changed assumptions, or adverse scenarios. Because target quantities are unobservable by construction, we design an evaluation framework based on necessary conditions any reliable estimator must satisfy: accuracy and calibration on verifiable proxies, and sensitivity to evidence. Agreement with expert panels and reasoning quality serve as corroboration. Across two domains (AI-augmented cybersecurity risk, ice-sheet contribution to sea-level rise), three benchmarks, and three reproduced expert studies, the estimates pass these tests: they improve systematically as evidence is added, agree with expert panels on most quantities, and correlate strongly with ground truth (Pearson r=0.91-0.98).

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文