AI 中文总结
本文提出QUEST系统，将庇护法申请裁决的可信度因素提取转化为信息检索任务，结合合成查询生成等技术，经评估发现该任务难度较高。
AI 中文摘要
庇护申请的法律裁决由冗长、复杂且异构的文档构成，涵盖申请人叙述性访谈、原始裁决及额外佐证材料。若申请被驳回，上诉程序中的关键问题是原始申请中的信息可信度是否为判定原始裁决的因素。本文提出QUEST系统（查询与主题提取系统），用于在两个丹麦庇护申请上诉数据集里提取并识别与可信度评估相关的因素。QUEST将该问题建模为信息检索任务，结合合成查询生成、主题提取与相关性评估，以识别上诉委员会申请材料中与可信度指标相关的信息。除标准检索评估指标外，本文提出一种不同于传统相关性的新型领域特定评估方式，以评估被测系统在可信度因素方面的性能，从而深入了解自动方法针对庇护上诉中不同类型指标返回答案的效果。结果表明，使用基于可信度的相关性评估来估计性能时，挑战会增加，凸显了该任务的难度。
英文摘要
Legal decisions on asylum applications consist of long, complex, and heterogeneous documents, covering narrative applicant interviews, original decisions, and additional supporting materials. If an application is rejected, a critical question in processing an appeal is whether the credibility of the information in the original application was a factor that determined the original decision. In this paper, we present the QUEST system (Query and Extraction System for Topics) to extract and identify factors relating to credibility assessments in two datasets of Danish asylum application appeals. QUEST frames this problem as an information retrieval task, combining synthetic query generation, topic extraction, and relevance assessment to identify information related to credibility indicators in appeals board application materials. In addition to standard retrieval evaluation metrics, we propose a new type of domain-specific assessments distinct from the traditional relevance to evaluate the performance of the tested systems with respect to credibility factors. In this way, we obtain insights about how well automatic methods can return answers for different types of indicators appearing in asylum appeals. Our results indicate that there is an increased challenge when estimating performance using credibility-based relevance assessments, thus pointing to the difficulty of the task.
