用于复杂评估任务的推理语言模型:从儿童保护案件报告中评估父母合作
Reasoning Language Models for complex assessments tasks: Evaluating parental cooperation from child protection case reports
AI总结:
本文研究了推理语言模型在评估儿童保护案件中父母合作方面的有效性,发现模型在母亲合作评估上表现更佳,但对父亲合作的评估准确性较低,提示需关注性别差异。
摘要:
目的:推理语言模型(RLMs)在解决复杂推理任务方面取得了显著进展。我们探讨了它们在 CPS 干预期间通过案件报告评估父母合作的潜力,这是一种信息模糊且矛盾的案件因素。方法:开发了一个四阶段工作流程,包括(1)案件报告的收集,(2)基于推理的父母合作评估,(3)自动类别提取,以及(4)案件标注。比较了不同参数规模(255B、32B、4B)的 RLMs 的性能,与人工验证的数据。两名专家人工评审员(EHRs)独立分类了加权随机样本的报告。结果:最大的 RLM 达到了最高的准确率(89%),优于初始方法(80%)。分类准确率在母亲(93%)方面高于父亲(85%),EHRs 也表现出相似的差异。结论:RLMs 的推理能够有效评估复杂的案件因素,如父母合作。对父亲合作评估的较低准确性支持了 CPS 干预中对母亲有更强专业关注的论点。
英文摘要:
Purpose: Reasoning language models (RLMs) have demonstrated significant advances in solving complex reasoning tasks. We examined their potential to assess parental cooperation during CPS interventions using case reports, a case factor characterized by ambiguous and conflicting information. Methods: A four stage workflow comprising (1) case reports collection, (2) reasoning-based assessment of parental cooperation, (3) automated category extraction, and (4) case labeling was developed. The performance of RLMs with different parameter sizes (255B, 32B, 4B) was compared against human validated data. Two expert human reviewers (EHRs) independently classified a weighted random sample of reports. Results: The largest RLM achieved the highest accuracy (89%), outperforming the initial approach (80%). Classification accuracy was higher for mothers (93%) than for fathers (85%), and EHRs exhibited similar differences. Conclusions: RLMs' reasoning can effectively assess complex case factors such as parental cooperation. Lower accuracy in assessing fathers' cooperation supports the argument of a stronger professional focus on mothers in CPS interventions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。