太有帮助、太无害、太诚实，还是恰到好处？
Too Helpful, Too Harmless, Too Honest or Just Right?
AI总结：
提出模块化对齐框架TrinityX，通过校准专家混合机制整合有帮助、无害和诚实三个维度，显著提升对齐性能并降低资源开销。
摘要：
大型语言模型（LLMs）在广泛的NLP任务中表现出色，但将其输出与有帮助、无害和诚实（HHH）原则对齐仍是一个持续的挑战。现有方法通常孤立地优化单一对齐维度，导致权衡和不一致的行为。尽管混合专家架构提供了模块化，但它们存在路由校准不良的问题，限制了对齐任务的有效性。我们提出TrinityX，这是一个模块化对齐框架，在Transformer架构中引入了校准专家混合。TrinityX为每个HHH维度利用单独训练的专家，通过校准的、任务自适应的路由机制整合它们的输出，将专家信号结合成统一的对齐感知表示。在三个标准对齐基准上的广泛实验——Alpaca（有帮助）、BeaverTails（无害）和TruthfulQA（诚实）——表明TrinityX超越了强基线，在胜率、安全分数和真实性上分别实现了32.5%、33.9%和28.4%的相对提升。此外，与先前基于MoE的方法相比，TrinityX将内存使用和推理延迟降低了40%以上。消融研究突出了校准路由的重要性，跨模型评估证实了TrinityX在不同LLM骨干上的泛化能力。
英文摘要：
Large Language Models (LLMs) exhibit strong performance across a wide range of NLP tasks, yet aligning their outputs with the principles of Helpfulness, Harmlessness, and Honesty (HHH) remains a persistent challenge. Existing methods often optimize for individual alignment dimensions in isolation, leading to trade-offs and inconsistent behavior. While Mixture-of-Experts (MoE) architectures offer modularity, they suffer from poorly calibrated routing, limiting their effectiveness in alignment tasks. We propose TrinityX, a modular alignment framework that incorporates a Mixture of Calibrated Experts (MoCaE) within the Transformer architecture. TrinityX leverages separately trained experts for each HHH dimension, integrating their outputs through a calibrated, task-adaptive routing mechanism that combines expert signals into a unified, alignment-aware representation. Extensive experiments on three standard alignment benchmarks-Alpaca (Helpfulness), BeaverTails (Harmlessness), and TruthfulQA (Honesty)-demonstrate that TrinityX outperforms strong baselines, achieving relative improvements of 32.5% in win rate, 33.9% in safety score, and 28.4% in truthfulness. In addition, TrinityX reduces memory usage and inference latency by over 40% compared to prior MoE-based approaches. Ablation studies highlight the importance of calibrated routing, and cross-model evaluations confirm TrinityX's generalization across diverse LLM backbones.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
