自我验证就是通过日本司法考试所需
Self-Verification is All You Need To Pass The Japanese Bar Examination
AI总结：
本文提出了一种自我验证模型，通过忠实复制考试格式和评分尺度，首次实现了LLM通过日本司法考试，无需修改原始问题结构或评分规则。
摘要：
尽管大型语言模型（LLMs）在快速进步，但实现可靠性能在高度专业和结构化的考试中仍是一个重大挑战。日本司法考试是一个特别具有挑战性的基准，要求不仅有先进的法律推理，还必须严格遵守复杂的答案格式，这些格式涉及对多个命题的联合评估。尽管最近的研究报告称通过将此类问题分解为更简单的真假判断来取得进步，但这些方法尚未在原始考试格式和评分方案下得到系统评估，留下了一个问题：它们是否真正捕捉到了考试级别的能力。在本文中，我们提出了一种自我验证模型，该模型在新构建的数据集上进行训练，该数据集忠实复制了考试的原始格式和评估尺度。我们的模型在实际考试尺度上能够超过官方通过分数，标志着我们所知的第一例LLM通过日本司法考试，而无需改变其原始问题结构或评分规则。我们进一步对替代策略进行了广泛比较，包括多代理推理和基于分解的监督，并发现这些方法未能达到可比的性能。我们的结果强调了格式忠实监督和一致性验证的重要性，并表明精心设计的单模型方法在高风险专业推理任务中可以优于更复杂的系统。我们的数据集和代码已公开可用。
英文摘要：
Despite rapid advances in large language models (LLMs), achieving reliable performance on highly professional and structured examinations remains a significant challenge. The Japanese bar examination is a particularly demanding benchmark, requiring not only advanced legal reasoning but also strict adherence to complex answer formats that involve joint evaluation of multiple propositions. While recent studies have reported improvements by decomposing such questions into simpler true--false judgments, these approaches have not been systematically evaluated under the original exam format and scoring scheme, leaving open the question of whether they truly capture exam-level competence. In this paper, we present a self-verification model trained on a newly constructed dataset that faithfully replicates the authentic format and evaluation scale of the exam. Our model is able to exceed the official passing score when evaluated on the actual exam scale, marking the first demonstration, to our knowledge, of an LLM passing the Japanese bar examination without altering its original question structure or scoring rules. We further conduct extensive comparisons with alternative strategies, including multi-agent inference and decomposition-based supervision, and find that these methods fail to achieve comparable performance. Our results highlight the importance of format-faithful supervision and consistency verification, and suggest that carefully designed single-model approaches can outperform more complex systems in high-stakes professional reasoning tasks. Our dataset and codes are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
