当驾驭胜过规模，以及当阅读胜过两者
When Harness Beats Scale, and When Reading Beats Both
浏览论文内容
中文总结 AI 辅助
本研究分析DocSem任务中系统在标注数据成功而测试集失败的原因，发现架构比规模更关键，且阅读质量而非推理能力决定了排行榜的区分。
中文摘要 AI 辅助
我们描述了我们在DocInsights 2026的DocSem文档基础定量推理共享任务中的系统，并分析了为什么它在标注数据上成功而在测试集上失败。该流程将混合块检索与在沙盒解释器中执行的Program-of-Thoughts（PoT）生成、自一致性采样以及从块级知识图谱中的实体增强相结合。在我们保留的划分上，应用架构对指标的影响远大于模型规模：PoT为紧凑的7B模型增加了0.282的联合准确率，但对72B模型最多只增加了0.005；而配备完整流程的27B模型与72B模型相当（0.884对0.873），参数减少了约2.7倍，二氧化碳排放量仅为四分之一。我们通过世界知识与语言知识之间的区别来解读这一结果，前者随参数急剧扩展，后者则平缓扩展，并表明结构化输出训练使紧凑模型具备流程就绪能力，而不仅仅是规模小。在栅格化的、带水印的测试PDF上，同一系统崩溃至13.58%的联合准确率（在163个队伍中排名第149）；对验证集进行受控的重新渲染重现了崩溃的OCR部分，同时界定了模拟遗漏的内容。审计评估输入的物理性质先于架构，而排行榜的双峰性与阅读质量（而非推理）将领域区分开来这一现象一致。
英文摘要
We describe our system for DocSem, the document-grounded quantitative reasoning shared task at DocInsights 2026, and analyze why it succeeded on labeled data and failed on the test set. The pipeline pairs hybrid block retrieval with Program-of-Thoughts (PoT) generation executed in a sandboxed interpreter, self-consistency sampling, and entity enrichment from chunk-level knowledge graphs. On our held-out split, application architecture moved the metrics far more than model scale did: PoT added 0.282 joint accuracy to a compact 7B model but at most 0.005 to a 72B model, and a 27B model with the full harness matched the 72B (0.884 vs.\ 0.873) at roughly 2.7$\times$ fewer parameters and a quarter of the CO$_2$. We read this through a distinction between world knowledge, which scales steeply with parameters, and language knowledge, which scales gently, and show that structured-output training makes a compact model harness-ready rather than merely small. On the raster, watermarked test PDFs the same system collapsed to 13.58\% joint (rank 149 of 163); a controlled re-rendering of the validation set reproduces the OCR half of the collapse while bounding what the simulation misses. Auditing the physical nature of evaluation inputs precedes architecture, and the leaderboard's bimodality is consistent with reading quality, not reasoning, having separated the field.
发表机构
- Novosibirsk State University（新西伯利亚国立大学）
- ITMO University（ITMO大学）
机构由 AI 辅助整理，请以论文原文为准。
