arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.11398cs.CLcs.AI
收藏

比较之前的推理:面向领域专用文本分析的LLM增强语义相似度度量

Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis

School of Computing, University of Georgia(佐治亚大学计算学院);School of Computer Science and Engineering, South China University of Technology(华南理工大学计算机科学与工程学院);School of Data Science, University of Virginia(弗吉尼亚大学数据科学学院);University of Georgia College of Pharmacy(佐治亚大学药学院);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院)

Shaochen Xu, Zihao Wu, Huaqin Zhao, Peng Shu, Zhengliang Liu, Wenxiong Liao, Sheng Li, Andrea Sikora, Tianming Liu, Xiang Li

AI总结:

本文提出利用GPT-4等LLM为零样本生成放射学报告标签并据此度量文本相似度,在MIMIC数据上验证其优于ROUGE和BLEU等传统NLP指标,展示了LLM半定量推理在专业领域语义分析中的潜力。

摘要:

在本研究中,我们利用LLM来增强语义分析并开发文本相似度度量,以解决传统无监督NLP指标(如ROUGE和BLEU)的局限性。我们开发了一个框架,其中使用GPT-4等LLM对放射学报告进行零样本文本识别和标签生成,然后将这些标签用作文本相似度的测量依据。通过在MIMIC数据上测试所提出的框架,我们发现GPT-4生成的标签能够显著改善语义相似度评估,其得分比传统NLP指标更贴近临床金标准。我们的工作证明了利用LLM的半定量推理结果对高度专业化领域的文本数据进行语义分析的可行性。虽然该框架是为放射学报告相似度分析而实现的,但其概念也可以扩展到其他专业领域。

英文摘要:

In this study, we leverage LLM to enhance the semantic analysis and develop similarity metrics for texts, addressing the limitations of traditional unsupervised NLP metrics like ROUGE and BLEU. We develop a framework where LLMs such as GPT-4 are employed for zero-shot text identification and label generation for radiology reports, where the labels are then used as measurements for text similarity. By testing the proposed framework on the MIMIC data, we find that GPT-4 generated labels can significantly improve the semantic similarity assessment, with scores more closely aligned with clinical ground truth than traditional NLP metrics. Our work demonstrates the possibility of conducting semantic analysis of the text data using semi-quantitative reasoning results by the LLMs for highly specialized domains. While the framework is implemented for radiology report similarity analysis, its concept can be extended to other specialized domains as well.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑