arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.02862cs.AIcs.LG
收藏

STEER: 通过受约束的质量多样性搜索实现推理时间的风险控制

STEER: Inference-Time Risk Control via Constrained Quality-Diversity Search

Eric Yang, Jong Ha Lee, Jonathan Amar, Elissa Ye, Yugang Jia

AI总结:

STEER通过受约束的质量多样性搜索实现推理时间的风险控制,提供可调节的决策保守性调整,提升临床分诊等场景下的行为覆盖和准确性。

摘要:

大型语言模型(LLMs)在为平均正确性训练时,往往会出现模式崩溃,导致在任务中多个合理响应可能存在的场景下产生狭窄的决策行为。这一限制在顺序决策设置中尤为突出,例如临床分诊,其中标准对齐方法会移除根据上下文约束调整特异度和灵敏度(ROC操作点)的能力。我们提出了STEER(通过进化群体精炼实现可调节的调优),这是一个无需训练的框架,重新引入了这种可调节的控制。STEER通过一种离线的受约束质量多样性搜索构建自然语言人物群体,该搜索在促进行为覆盖的同时,强制最低的安全性、推理和稳定性阈值。在推理时间,STEER暴露了一个单一、可解释的控制参数,该参数将用户指定的风险百分位映射到选定的人物,从而产生决策保守性的单调调整。在两个临床分诊基准测试中,STEER在行为覆盖方面比基于温度的采样和静态人物群体更广。与代表性的后训练方法相比,STEER在明确的紧急案例上保持了显著更高的准确性,同时在模糊决策上提供了可比的控制。这些结果证明STEER作为一种安全保持的范式,能够实现风险控制,能够在不损害领域能力的情况下引导行为。

英文摘要:

Large Language Models (LLMs) trained for average correctness often exhibit mode collapse, producing narrow decision behaviors on tasks where multiple responses may be reasonable. This limitation is particularly problematic in ordinal decision settings such as clinical triage, where standard alignment removes the ability to trade off specificity and sensitivity (the ROC operating point) based on contextual constraints. We propose STEER (Steerable Tuning via Evolutionary Ensemble Refinement), a training-free framework that reintroduces this tunable control. STEER constructs a population of natural-language personas through an offline, constrained quality-diversity search that promotes behavioral coverage while enforcing minimum safety, reasoning, and stability thresholds. At inference time, STEER exposes a single, interpretable control parameter that maps a user-specified risk percentile to a selected persona, yielding a monotonic adjustment of decision conservativeness. On two clinical triage benchmarks, STEER achieves broader behavioral coverage compared to temperature-based sampling and static persona ensembles. Compared to a representative post-training method, STEER maintains substantially higher accuracy on unambiguous urgent cases while providing comparable control over ambiguous decisions. These results demonstrate STEER as a safety-preserving paradigm for risk control, capable of steering behavior without compromising domain competence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑