当模型说'无话可说'时，我们得知有帮助性已死，诚实仍活着，安全性却感到害怕
When the Model Said 'No Comment', We Knew Helpfulness Was Dead, Honesty Was Alive, and Safety Was Terrified
AI总结：
AlignX通过两阶段框架解决LLM多目标对齐问题，提升有帮助性、无害性和诚实性，同时减少延迟和内存使用。
摘要：
大型语言模型（LLMs）需要符合人类价值观——有帮助性、无害性和诚实性（HHH）——这对于安全部署至关重要。现有工作使用监督微调（SFT）和专家混合（MoE）来对齐LLMs。然而，这些工作在多目标设置中面临挑战，例如SFT导致冲突目标之间的干扰，而MoEs则遭受路由不准确的问题。我们称这种失败模式为轴塌陷，表现为（1）不相交的特征空间导致灾难性遗忘，以及（2）从误路由专家中得到不可靠的推理。为了解决这个问题，我们提出了AlignX，一个两阶段框架。第一阶段使用提示注入微调来提取轴特定任务特征，缓解灾难性遗忘。第二阶段部署了一个MoCaE模块，利用分形和自然几何校准专家路由，提高推理可靠性。AlignX在Alpaca（有帮助性）、BeaverTails（无害性）和TruthfulQA（诚实性）上实现了显著提升，胜率增加171.5%，真实性-信息量增加110.1%，安全违规减少4.3%。它还比先前的MoEs减少了超过35%的延迟和内存使用。在四个LLM上的结果验证了其通用性。
英文摘要：
Large Language Models (LLMs) need to be in accordance with human values-being helpful, harmless, and honest (HHH)-is important for safe deployment. Existing works use Supervised Fine-Tuning (SFT) and Mixture-of-Experts (MoE) to align LLMs. However, these works face challenges in multi-objective settings, such as SFT leading to interference between conflicting objectives, while MoEs suffer from miscalibrated routing. We term this failure mode Axis Collapse, marked by (1) disjoint feature spaces causing catastrophic forgetting, and (2) unreliable inference from misrouted experts. To resolve this, we propose AlignX, a two-stage framework. Stage 1 uses prompt-injected fine-tuning to extract axis-specific task features, mitigating catastrophic forgetting. Stage 2 deploys a MoCaE module that calibrates expert routing using fractal and natural geometry, improving inference reliability. AlignX achieves significant gains on Alpaca (Helpfulness), BeaverTails (Harmlessness), and TruthfulQA (Honesty), with +171.5% win rate, +110.1% in truthfulness-informativeness, and 4.3% fewer safety violations. It also reduces latency and memory usage by over 35% compared to prior MoEs. Results across four LLMs validate its generalizability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
