FMBench: 自适应大语言模型输出格式化
FMBench: Adaptive Large Language Model Output Formatting
AI总结：
FMBench通过结合监督微调和强化学习微调，提升大语言模型在Markdown格式化任务中的语义和结构准确性。
摘要：
生成同时满足语义意图和格式约束的输出对于将大语言模型部署到面向用户和系统集成的工作流中至关重要。在本工作中，我们专注于Markdown格式化，它在助手、文档和工具增强的管道中普遍存在，但仍容易产生微妙且难以检测的错误（例如，损坏的列表、格式错误的表格、不一致的标题和无效的代码块），这些错误会显著降低下游的可用性。我们提出了FMBench，这是一个用于自适应Markdown输出格式化的基准，它在广泛的遵循指示的场景中评估模型，这些场景具有多样的结构要求。FMBench强调现实中的格式化行为，如多层次组织、混合内容（自然语言与列表/表格/代码交错）以及严格遵守用户指定的布局约束。为了在不依赖严格解码约束的情况下提高Markdown合规性，我们提出了一种轻量级的对齐管道，结合了监督微调（SFT）与强化学习微调。从基础模型开始，我们首先在指令-响应对上进行SFT，然后优化一个平衡语义忠实度和结构正确性的复合目标。在两个模型家族（OpenPangu和Qwen）上的实验表明，SFT一致地提高了语义对齐性，而强化学习在初始化自强SFT策略时提供了在挑战性Markdown指令上的额外增益。我们的结果还揭示了语义和结构目标之间固有的权衡，突显了精心设计的奖励在可靠格式化生成中的重要性。代码可在：https://github.com/FudanCVL/FMBench。
英文摘要：
Producing outputs that satisfy both semantic intent and format constraints is essential for deploying large language models in user-facing and system-integrated workflows. In this work, we focus on Markdown formatting, which is ubiquitous in assistants, documentation, and tool-augmented pipelines but still prone to subtle, hard-to-detect errors (e.g., broken lists, malformed tables, inconsistent headings, and invalid code blocks) that can significantly degrade downstream usability. We present FMBench, a benchmark for adaptive Markdown output formatting that evaluates models under a wide range of instruction-following scenarios with diverse structural requirements. FMBench emphasizes real-world formatting behaviors such as multi-level organization, mixed content (natural language interleaved with lists/tables/code), and strict adherence to user-specified layout constraints. To improve Markdown compliance without relying on hard decoding constraints, we propose a lightweight alignment pipeline that combines supervised fine-tuning (SFT) with reinforcement learning fine-tuning. Starting from a base model, we first perform SFT on instruction-response pairs, and then optimize a composite objective that balances semantic fidelity with structural correctness. Experiments on two model families (OpenPangu and Qwen) show that SFT consistently improves semantic alignment, while reinforcement learning provides additional gains in robustness to challenging Markdown instructions when initialized from a strong SFT policy. Our results also reveal an inherent trade-off between semantic and structural objectives, highlighting the importance of carefully designed rewards for reliable formatted generation. Code is available at: https://github.com/FudanCVL/FMBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
