arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2104.13841cs.CLcs.IR
收藏

基于内容的法律文献推荐中的文献表示评估

Evaluating Document Representations for Content-based Legal Literature Recommendations

Open Legal Data(开放法律数据);German Research Center for Artificial Intelligence(德国人工智能研究中心);ETH Zurich(苏黎世联邦理工学院);University of Wuppertal(伍珀塔尔大学)

Malte Ostendorff, Elliott Ash, Terry Ruas, Bela Gipp, Julian Moreno-Schneider, Georg Rehm

摘要:

推荐系统帮助法律专业人士找到相关文献来支持他们的案件。尽管它的重要性的专业,法律应用并没有反映在推荐系统和表示学习研究的最新进展。同时,法律推荐系统通常在没有任何公共基准数据集的情况下在小规模用户研究中进行评估。因此,这些研究的重现性有限。为了解决研究与实践之间的差距,我们探索了一套国家的最先进的文件表示方法检索语义相关的美国判例法的任务。我们评估基于文本的(例如,fastText、Transformers)、基于引用的(例如,DeepWalk,Poincaré)和混合方法。我们使用两种银标准对2,964份文件进行了注释,共比较了27种方法。银标准是从Open Case Book和Wikispecies新创建的,可以在开放许可下重复使用,以促进可重复性。我们的实验表明,来自平均fastText单词向量(在法律语料库上训练)的文档表示产生了最好的结果,紧随其后的是庞加莱引用嵌入。以混合方式组合fastText和Poincaré进一步改善了整体结果。除了整体性能外,我们还根据文档长度,引用计数和建议的覆盖范围分析了这些方法。我们在https://github.com/malteos/legal-document-similarity/上公开我们的源代码、模型和数据集。

英文摘要:

Recommender systems assist legal professionals in finding relevant literature for supporting their case. Despite its importance for the profession, legal applications do not reflect the latest advances in recommender systems and representation learning research. Simultaneously, legal recommender systems are typically evaluated in small-scale user study without any public available benchmark datasets. Thus, these studies have limited reproducibility. To address the gap between research and practice, we explore a set of state-of-the-art document representation methods for the task of retrieving semantically related US case law. We evaluate text-based (e.g., fastText, Transformers), citation-based (e.g., DeepWalk, Poincaré), and hybrid methods. We compare in total 27 methods using two silver standards with annotations for 2,964 documents. The silver standards are newly created from Open Case Book and Wikisource and can be reused under an open license facilitating reproducibility. Our experiments show that document representations from averaged fastText word vectors (trained on legal corpora) yield the best results, closely followed by Poincaré citation embeddings. Combining fastText and Poincaré in a hybrid manner further improves the overall result. Besides the overall performance, we analyze the methods depending on document length, citation count, and the coverage of their recommendations. We make our source code, models, and datasets publicly available at https://github.com/malteos/legal-document-similarity/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑