InfiMed-ORBIT: 通过基于评分标准的增量训练使大语言模型对齐开放复杂任务
InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training
AI总结：
提出ORBIT框架，利用动态生成的病例条件评分标准指导增量强化学习，仅用2k样本将Qwen3-4B-Instruct在HealthBench-Hard上的得分从7.0提升至27.5，达到同规模开源模型最优。
摘要：
强化学习（RL）推动了大语言模型（LLM）的许多近期突破，尤其是在奖励可自动计算的任务（如代码生成）中。然而，在开放式的医学对话中，RL效果较差，因为反馈模糊、依赖上下文，且难以简单总结为单一标量信号——通常需要高度监督的奖励模型，并存在奖励破解的风险。因此，我们引入了ORBIT，一个专为关键医学对话设计的基于评分标准的开放式增量训练框架。ORBIT将医学对话构建与动态生成的病例条件评分标准相结合，这些评分标准作为增量RL的自适应指南。与依赖外部医学知识库或手工规则的方法不同，ORBIT使用评分标准引导的评估，并可与通用指令遵循LLM一起实现，避免了任务特定的评判微调。仅使用2k训练样本，ORBIT将Qwen3-4B-Instruct的HealthBench-Hard得分从7.0提升至27.5，在相似规模的开源模型中实现了最先进的性能，同时随着评分标准覆盖范围的扩大，保持了良好的咨询质量。
英文摘要：
Reinforcement learning (RL) has powered many recent breakthroughs in large language models (LLMs), especially for tasks where rewards can be computed automatically, such as code generation. However, it is less effective in open-ended medical dialogue, where feedback is ambiguous, context-dependent, and difficult to simply summarize into a single scalar signal-often requiring heavily supervised reward models and creating risks of reward hacking. Thus, we introduce ORBIT, an open-ended rubric-based incremental training framework tailored for critical medical dialogues. ORBIT integrates medical dialogue construction with dynamically generated case-conditioned rubrics that serve as adaptive guides for incremental RL. Unlike approaches that rely on external medical knowledge bases or handcrafted rules, ORBIT uses rubric-guided evaluation and can be implemented with general-purpose instruction-following LLMs, avoiding task-specific judge fine-tuning. With only 2k training samples, ORBIT raises Qwen3-4B-Instruct's HealthBench-Hard score from 7.0 to 27.5, achieving state-of-the-art performance among similarly sized open-source models while maintaining strong consultation quality as rubric coverage broadens.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
