arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.15313cs.IRcs.CL

STARD:一个包含非专业人士真实查询的中文法条检索数据集

STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionals

Tsinghua University(清华大学);Peking University(北京大学)

Weihang Su, Yiran Hu, Anzhe Xie, Qingyao Ai, Zibing Que, Ning Zheng, Yun Liu, Weixing Shen, Yiqun Liu

AI总结:

针对现有法条检索基准忽视普通公众非专业查询的问题,研究团队构建了包含1543个真实咨询查询、55348篇候选法条的中文数据集STARD,评估发现现有检索方法在该数据集上表现不佳,为相关研究提供了新基准。

摘要:

法条检索旨在为特定查询找到相关的法律条文,这一过程是法律咨询、自动化司法决策、法律文书起草等广泛法律应用的基础。现有法条检索基准聚焦于来自律师考试、法律案件文书等来源的正式专业查询,忽视了普通公众的非专业查询——这类查询往往缺乏精准的法律术语和法条引用。为填补这一空白,我们提出了STAtute Retrieval Dataset(STARD,法条检索数据集),这是一个中文数据集,包含从真实法律咨询中收集的1543个查询案例以及55348篇候选法条。与现有主要聚焦专业法律查询的法条检索数据集不同,STARD捕捉了普通公众真实查询的复杂性与多样性。通过对多种检索基线的综合评估,我们发现现有检索方法在应对非专业用户提出的这些真实查询时均存在不足,最优方法的Recall@100仅达到0.907,表明该领域仍需进一步探索与更多研究。所有代码和数据集可在以下地址获取:https://github.com/oneal2000/STARD/tree/main

英文摘要:

Statute retrieval aims to find relevant statutory articles for specific queries. This process is the basis of a wide range of legal applications such as legal advice, automated judicial decisions, legal document drafting, etc. Existing statute retrieval benchmarks focus on formal and professional queries from sources like bar exams and legal case documents, thereby neglecting non-professional queries from the general public, which often lack precise legal terminology and references. To address this gap, we introduce the STAtute Retrieval Dataset (STARD), a Chinese dataset comprising 1,543 query cases collected from real-world legal consultations and 55,348 candidate statutory articles. Unlike existing statute retrieval datasets, which primarily focus on professional legal queries, STARD captures the complexity and diversity of real queries from the general public. Through a comprehensive evaluation of various retrieval baselines, we reveal that existing retrieval approaches all fall short of these real queries issued by non-professional users. The best method only achieves a Recall@100 of 0.907, suggesting the necessity for further exploration and additional research in this area. All the codes and datasets are available at: https://github.com/oneal2000/STARD/tree/main

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑