评估细调LLM模型在医疗转录中的表现：针对小规模低资源语言验证数据集
Evaluating Fine-Tuned LLM Model For Medical Transcription With Small Low-Resource Languages Validated Dataset
AI总结：
本研究通过在有限的芬兰语验证数据集上微调LLaMA 3.1-8B，评估了领域对齐的NLP模型在医疗转录中的有效性，结果显示BLEU、ROUGE-L和BERTScore F1等指标表现良好，表明细调在芬兰语医疗口语转录中的可行性。
摘要：
临床文档是患者安全、诊断和连续护理的关键因素。电子健康记录（EHR）的行政负担是医生倦怠的重要因素。这对低资源语言，包括芬兰语，是一个关键问题。本研究旨在通过在Metropolia应用科学大学学生模拟临床对话的有限验证语料库上微调LLaMA 3.1-8B，调查领域对齐的自然语言处理（NLP）模型在芬兰语医疗转录中的有效性。医疗转录的微调过程使用了受控的预处理和优化方法。通过七折交叉验证评估微调效果。微调后的LLaMA 3.1-8B的评估指标为BLEU=0.1214，ROUGE-L=0.4982，BERTScore F1=0.8230。结果显示n-gram重叠较低但与参考转录有强语义相似性。本研究表明，微调在芬兰语医疗口语转录中是有效的，并支持在芬兰语中微调隐私导向的领域专用大语言模型用于临床文档的可行性。此外，还为未来工作提供了方向。
英文摘要：
Clinical documentation is a critical factor for patient safety, diagnosis, and continuity of care. The administrative burden of EHRs is a significant factor in physician burnout. This is a critical issue for low-resource languages, including Finnish. This study aims to investigate the effectiveness of a domain-aligned natural language processing (NLP); large language model for medical transcription in Finnish by fine-tuning LLaMA 3.1-8B on a small validated corpus of simulated clinical conversations by students at Metropolia University of Applied Sciences. The fine-tuning process for medical transcription used a controlled preprocessing and optimization approach. The fine-tuning effectiveness was evaluated by sevenfold cross-validation. The evaluation metrics for fine-tuned LLaMA 3.1-8B were BLEU = 0.1214, ROUGE-L = 0.4982, and BERTScore F1 = 0.8230. The results showed a low n-gram overlap but a strong semantic similarity with reference transcripts. This study indicate that fine-tuning can be an effective approach for translation of medical discourse in spoken Finnish and support the feasibility of fine-tuning a privacy-oriented domain-specific large language model for clinical documentation in Finnish. Beside that provide directions for future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
