窥视LLM内部：利用LLM的内部构件增强法律分类的可靠性
Peeking Inside LLMs: Leveraging Internal Artifacts of LLMs for Enhancing Reliability in Legal Classification
AI总结：
针对LLM在法律领域易产生错误或幻觉的问题，提出利用模型内部构件特征构建下游分类器检测预测正确性，在保释决策和法规违规预测任务上验证了有效性。
摘要：
大型语言模型（LLM）越来越多地被应用于法律领域。然而，尽管性能强大，LLM容易产生错误或幻觉输出，这引发了对其在高风险领域（如法律）中可靠性的严重担忧。因此，检测基于LLM的系统响应的正确性是一个关键挑战。在这项工作中，我们探索了利用LLM的内部构件来检测其在法律领域分类任务中预测正确性的潜力。我们开发了利用这些内部构件特征的方法，构建能够识别错误LLM输出的下游分类器。我们在两个代表性的法律分类任务上评估了我们的方法：保释决策预测和法规违规预测。我们的实验结果表明，LLM的内部构件是检测法律分类任务中错误预测的可靠指标，并且可以应用于增强基于LLM的分类系统的可靠性。
英文摘要：
Large Language Models (LLMs) are increasingly being adopted in the legal domain. However, despite their strong performance, LLMs are prone to generating incorrect or hallucinated outputs, raising serious concerns about their reliability in high-stakes domains such as law. Detecting the correctness of responses of LLM-based systems is therefore a critical challenge. In this work, we explore the potential of leveraging internal artifacts of LLM to detect the correctness of their predictions in legal-domain classification tasks. We develop approaches that utilize features derived from these internal artifacts to build downstream classifiers capable of identifying incorrect LLM outputs. We evaluate our approach on two representative legal classification tasks: bail decision prediction and statute violation prediction. Our experimental results demonstrate that LLMs' internal artifacts are reliable indicators for detecting incorrect predictions in legal classification tasks, and can be applied to enhance the reliability of LLM-based classification systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
