arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.12639cs.CL
收藏

CLASE:一种中文法律文本风格评估的混合方法

CLASE: A Hybrid Method for Chinese Legalese Stylistic Evaluation

Nanyang Technological University(南洋理工大学)

Yiran Rex Ma, Yuxiao Ye, Huiyuan Xie

AI总结:

本文提出CLASE混合方法,通过结合语言特征和经验指导的LLM评估,提升法律文本风格质量,实验表明其比传统方法更符合人类判断。

摘要:

大型语言模型生成的法律文本通常能实现合理的事实准确性,但往往无法遵循法律写作的专门风格规范和语言惯例。为提高风格质量,建立可靠的评估方法是关键第一步。然而,让法律专家手动开发此类指标不现实,因为法律写作中的隐含风格要求难以明确化为明确的评分标准。同时,现有自动评估方法也存在不足:基于参考的指标将语义准确性与风格忠实度混为一谈,而LLM作为评判者的方法则存在不透明和不一致的问题。为解决这些挑战,我们引入CLASE(中文法律风格评估),一种专注于法律文本风格表现的混合评估方法。该方法结合了基于语言特征的评分和经验指导的LLM作为评判者评分。特征系数和LLM评分经验均从真实法律文件与其LLM恢复版本的对比对中学习。这种混合设计以透明且无参考的方式捕捉了表层特征和隐含的风格规范。在200份中文法律文件上的实验表明,CLASE在与人类判断的一致性方面显著优于传统指标和纯LLM作为评判者方法。除了改进的一致性,CLASE还提供可解释的评分分解和改进建议,为法律文本生成中的专业风格评估提供了可扩展和实用的解决方案(CLASE的代码和数据可在:https://github.com/rexera/CLASE获得)。

英文摘要:

Legal text generated by large language models (LLMs) can usually achieve reasonable factual accuracy, but it frequently fails to adhere to the specialised stylistic norms and linguistic conventions of legal writing. In order to improve stylistic quality, a crucial first step is to establish a reliable evaluation method. However, having legal experts manually develop such a metric is impractical, as the implicit stylistic requirements in legal writing practice are difficult to formalise into explicit rubrics. Meanwhile, existing automatic evaluation methods also fall short: reference-based metrics conflate semantic accuracy with stylistic fidelity, and LLM-as-a-judge evaluations suffer from opacity and inconsistency. To address these challenges, we introduce CLASE (Chinese LegAlese Stylistic Evaluation), a hybrid evaluation method that focuses on the stylistic performance of legal text. The method incorporates a hybrid scoring mechanism that combines 1) linguistic feature-based scores and 2) experience-guided LLM-as-a-judge scores. Both the feature coefficients and the LLM scoring experiences are learned from contrastive pairs of authentic legal documents and their LLM-restored counterparts. This hybrid design captures both surface-level features and implicit stylistic norms in a transparent, reference-free manner. Experiments on 200 Chinese legal documents show that CLASE achieves substantially higher alignment with human judgments than traditional metrics and pure LLM-as-a-judge methods. Beyond improved alignment, CLASE provides interpretable score breakdowns and suggestions for improvements, offering a scalable and practical solution for professional stylistic evaluation in legal text generation (Code and data for CLASE is available at: https://github.com/rexera/CLASE).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑