LogiNumSynth：为语言模型合成联合逻辑-数值推理问题
LogiNumSynth: Synthesizing Joint Logical-Numerical Reasoning Problems for Language Models
AI总结：
提出LogiNumSynth问题合成器，通过细粒度控制生成联合逻辑-数值推理任务，支持过程与答案评估及针对性训练，有效诊断并提升语言模型的综合推理能力。
摘要：
联合逻辑-数值推理仍然是语言模型面临的一项重大挑战，然而现有数据集依赖于固定的规则集，且对任务复杂度的控制有限，从而限制了其在评估和训练方面的泛化能力。我们提出了LogiNumSynth，这是一个灵活的自然语言问题合成器，能够合成需要熟练掌握联合逻辑推理（例如基于规则的推理）和数值推理（例如算术计算）的任务。LogiNumSynth支持对推理世界的丰富度、逻辑推理深度以及数值计算的复杂度进行细粒度控制，从而实现跨难度级别的灵活数据合成。我们展示了三项关键贡献：（1）合成器——在自然语言上合成完全可控的联合推理任务；（2）评估与过程分析——同时评估过程准确率和答案准确率；（3）针对性训练——利用合成数据增强LLMs的推理性能。使用多个LLMs进行的实验突显了其在逻辑-数值推理方面持续存在的弱点，表明LogiNumSynth既可以作为诊断工具，也可以作为针对性监督的来源，以推进综合推理技能。
英文摘要：
Joint logical-numerical reasoning remains a major challenge for language models, yet existing datasets rely on fixed rule sets and offer limited control over task complexity, constraining their generalizability for evaluation and training. We present LogiNumSynth, a flexible natural language problem synthesizer that synthesizes tasks requiring proficiency in joint logical reasoning (e.g., rule-based reasoning) and numerical reasoning (e.g., arithmetic computation). LogiNumSynth supports fine-grained control over reasoning world richness, logical reasoning depth, and the complexity of numerical computations, enabling flexible data synthesis across difficulty levels. We demonstrate three key contributions: (1) Synthesizer -- synthesizing fully controllable joint reasoning tasks over natural language; (2) Evaluation & Process Analysis -- evaluating both process accuracy and answer accuracy; (3) Targeted Training -- using synthesized data to enhance LLMs' reasoning performance. Experiments with multiple LLMs highlight persistent weaknesses in logical-numerical reasoning, showing that LogiNumSynth can serve as both a diagnostic tool and a source of targeted supervision for advancing integrated reasoning skills.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
