发表机构
Alibaba Group; Ant Group（阿里巴巴集团; 蚂蚁集团）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对人文社科领域的偏好对齐需求，提出BridgeAlign框架，经21万+合成偏好样本对齐后，使Qwen3-8B在17个基准测试中优于11个强基线，且人工偏好与知识能力无权衡。
AI 中文摘要
尽管大型语言模型（LLM）的数据合成已十分普遍，但该技术主要针对答案可验证的领域，忽视了人文社科（HSS）这类开放式领域——在该领域中，细微的质量判断比客观正确性更为重要。这使得偏好对齐成为适用于广泛人文社科任务的自然范式。然而现有方法要么成本高昂，要么未针对广泛的人文社科学科定制。因此我们提出BridgeAlign，它是首批面向广泛人文社科学科的偏好对齐流程，包含三个阶段：i）种子数据整理：通过启发式方法或基于LLM的过滤及文本优化，从网络语料库中整理人文社科种子文档；ii）偏好数据合成：通过基于角色设定的指令反转及问答一致性检查生成偏好三元组；iii）偏好优化：超越简单的人工与模型对比启发式方法，首先将偏好建立在人文社科质量标准上，再通过受控质量退化生成过渡响应，形成近边界偏好对以实现更精细的质量区分。在超过21万个合成偏好样本上进行对齐后，BridgeAlign使Qwen3-8B在17个基准测试中取得了优于11个强基线的平均表现；重要的是，它同时在人工偏好和基于知识的能力上领先，且两者之间不存在权衡，这得到了大量实验的支持并符合现有理论背景。
英文摘要
While data synthesis for large language models (LLMs) is prevalent, it primarily targets domains with verifiable answers, overlooking open-ended humanities and social sciences (HSS), where nuanced quality judgments matter more than objective correctness. This makes preference alignment a natural paradigm for broad HSS tasks. Yet existing methods are either costly or not tailored to broad HSS disciplines. We thus propose BridgeAlign, among the first preference-alignment pipelines for broad HSS disciplines, with three phases: i) Seed Curation: curating HSS seed documents from web corpora via heuristic/LLM-based filtering and text refinement; ii) Preference Data Synthesis: generating preference triplets via persona-based instruction inversion with Q&A consistency checks; iii) Preference Optimization: moving beyond naive human-vs-model heuristics by first grounding preferences in HSS quality rubric, then generating transitional responses via controlled quality degradation to form near-boundary preference pairs for finer-grained quality discrimination. Aligning over 210k synthetic preference samples, BridgeAlign enables Qwen3-8B to achieve the best average across 17 benchmarks against 11 strong baselines; importantly, leading on both human-preference and knowledge-based capabilities at once, with no trade-off between them, as supported by extensive experiments and contextualized by existing theories.
