arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.07539cs.CL
收藏

MAWARITH:面向大语言模型的法律继承推理数据集与基准

MAWARITH: A Dataset and Benchmark for Legal Inheritance Reasoning with LLMs

Hamad Bin Khalifa University, Qatar(卡塔尔哈马德比内卡菲大学);Nazarbayev University, Kazakhstan(哈萨克斯坦纳扎尔巴耶夫大学)

Abdessalam Bouchekif, Shahd Gaben, Samer Rashwani, Somaya Eltanbouly, Mutaz Al-Khatib, Heba Sbahi, Mohammed Ghaly, Emad Mohamed

AI总结:

提出MAWARITH数据集,包含12,500个阿拉伯语继承案例,支持端到端的伊斯兰继承推理,并设计MIR-E多阶段评估指标,实验显示商业模型达90%而开源模型低于50%。

摘要:

伊斯兰继承法对大语言模型具有挑战性,因为解决继承案件需要复杂、结构化、多步骤的推理以及正确应用法学规则来计算继承人的份额。我们引入了\textit{MAWARITH},一个大规模标注数据集,包含12,500个阿拉伯语继承案例,用于训练和评估模型的完整推理链:(i) 识别合格继承人,(ii) 应用阻断(\textit{\d{h}ajb})和分配规则,以及(iii) 计算精确继承份额。据我们所知,\textit{MAWARITH}是首个专为端到端伊斯兰继承推理设计的阿拉伯语语料库和基准。与之前将继承案件解决限制为多项选择题的数据集不同,\textit{MAWARITH}支持完整的推理链,并提供基于经典法学来源和既定继承规则的逐步解决方案及理由,以及精确的份额计算。这使得模型能够学习如何生成详细的、逐步响应用户查询的答案,反映现实世界的伊斯兰继承案例。为了超越最终答案准确性来评估模型,我们提出了\textit{MIR-E}(Mawarith继承推理评估),一个加权多阶段指标,对关键推理阶段进行评分,并捕捉整个流水线中的错误传播。我们在零样本设置下评估了六个大语言模型。一个商业模型达到了约90%,而所有评估的开源模型均低于50%。我们的错误分析识别了重复出现的失败模式,包括场景误解、继承人识别错误、份额分配错误以及关键继承规则(如\textit{\textquotesingle awl}和\textit{radd})的缺失或错误应用。\textit{MAWARITH}数据集在此https URL公开可用。

英文摘要:

Islamic inheritance law is challenging for large language models because solving inheritance cases requires complex, structured, multi-step reasoning and the correct application of juristic rules to compute heirs' shares. We introduce \textit{MAWARITH}, a large-scale annotated dataset of 12,500 Arabic inheritance cases for training and evaluating models on the full reasoning chain: (i) identifying eligible heirs, (ii) applying blocking (\textit{\d{h}ajb}) and allocation rules, and (iii) computing exact inheritance shares. To the best of our knowledge, \textit{MAWARITH} is the first Arabic corpus and benchmark designed for end-to-end Islamic inheritance reasoning. Unlike prior datasets that restrict inheritance case solving to multiple-choice questions, \textit{MAWARITH} supports the full reasoning chain and provides step-by-step solutions with justifications grounded in classical juristic sources and established inheritance rules, as well as exact share calculations. This enables models to learn how to generate detailed, step-by-step responses to user queries that reflect real-world Islamic inheritance cases. To evaluate models beyond final-answer accuracy, we propose \textit{MIR-E} (Mawarith Inheritance Reasoning Evaluation), a weighted multi-stage metric that scores key reasoning stages and captures error propagation across the pipeline. We evaluate six large language models in a zero-shot setting. A commercial model achieves about 90\%, whereas all evaluated open-source models remain below 50\%. Our error analysis identifies recurring failure patterns, including scenario misinterpretation, errors in heir identification, errors in share allocation, and missing or incorrect application of key inheritance rules such as \textit{\textquotesingle awl} and \textit{radd}. The \textit{MAWARITH} dataset is publicly available at https://gitlab.com/nlpresearcher/mawarith.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑