基于角色的领域特定文档摘要研究
On The Persona-based Summarization of Domain-Specific Documents
AI总结：
本文提出通过微调领域特定小型LLM并结合AI评判来生成基于角色的领域摘要，并验证AI评判与人工评判一致，从而高效低成本地扩展到多领域。
摘要：
在领域特定知识不断扩展的世界中，消费和存储信息的复杂性日益增加，因此需要从大型信息库中生成摘要。然而，一个领域中的每种角色对信息的需求不同，因而其摘要需求也不同。例如，在医疗健康领域，基于角色（如医生、护士、患者等）的方法对于高效传递有针对性的医疗信息至关重要。由人工对领域特定信息进行基于角色的摘要是一项高认知负荷任务，通常不被优先采用。两个不同的人生成的摘要具有高度可变性，并且随着领域和角色的增长，在成本和领域专业知识方面无法扩展。此外，使用通用大型语言模型（LLMs）生成的AI摘要，除非这些模型已经在领域特定数据上经过专门训练，否则不一定能为不同领域提供令人满意的准确性，而且在日常运营中使用也可能非常昂贵。本文的贡献有两点：1）我们提出了一种利用医疗语料库高效微调领域特定小型基础LLM的方法，并且表明我们可以使用基于AI的评判来有效评估摘要质量。2）我们进一步表明，基于AI的评判与基于人工的摘要评判具有良好的一致性。因此，这种生成领域特定、基于角色的摘要的AI流水线可以以非常高效且成本效益高的方式轻松扩展到其他领域，如法律、企业文档、教育等。
英文摘要：
In an ever-expanding world of domain-specific knowledge, the increasing complexity of consuming, and storing information necessitates the generation of summaries from large information repositories. However, every persona of a domain has different requirements of information and hence their summarization. For example, in the healthcare domain, a persona-based (such as Doctor, Nurse, Patient etc.) approach is imperative to deliver targeted medical information efficiently. Persona-based summarization of domain-specific information by humans is a high cognitive load task and is generally not preferred. The summaries generated by two different humans have high variability and do not scale in cost and subject matter expertise as domains and personas grow. Further, AI-generated summaries using generic Large Language Models (LLMs) may not necessarily offer satisfactory accuracy for different domains unless they have been specifically trained on domain-specific data and can also be very expensive to use in day-to-day operations. Our contribution in this paper is two-fold: 1) We present an approach to efficiently fine-tune a domain-specific small foundation LLM using a healthcare corpus and also show that we can effectively evaluate the summarization quality using AI-based critiquing. 2) We further show that AI-based critiquing has good concordance with Human-based critiquing of the summaries. Hence, such AI-based pipelines to generate domain-specific persona-based summaries can be easily scaled to other domains such as legal, enterprise documents, education etc. in a very efficient and cost-effective manner.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
