arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.12097cs.LGcs.AIcs.CL
收藏

利用LLM和网络分析从学术论文中提取研究目标、机器学习模型名称与数据集名称及其相互关系分析

Extraction of Research Objectives, Machine Learning Model Names, and Dataset Names from Academic Papers and Analysis of Their Interrelationships Using LLM and Network Analysis

Aichi Institute of Technology(爱知工业大学);University of Tokyo(东京大学);Hokkaido University(北海道大学);Nagaoka University of Technology(长冈技术科学大学);Saitama University(埼玉大学)

S. Nishio, H. Nonaka, N. Tsuchiya, A. Migita, Y. Banno, T. Hayashi, H. Sakaji, T. Sakumoto, K. Watabe

AI总结:

本研究提出利用LLM、嵌入模型和网络聚类从学术论文中提取任务、机器学习模型与数据集名称并分析其关系,使用Llama3在各类别F分数超0.8,并在金融领域验证了有效性。

摘要:

机器学习在各行业被广泛应用。为特定任务确定合适的机器学习模型和数据集对于机器学习的有效工业应用至关重要。然而,这需要机器学习和相关领域的专业知识,导致学习成本高昂。因此,专注于从学术论文中提取任务、机器学习模型和数据集组合的研究具有极其重要的意义,因为这能促进合适方法的自动推荐。从学术论文中提取信息的传统方法仅限于将机器学习模型和其他实体识别为命名实体。为解决这一问题,本研究提出了一种方法,利用LLM、嵌入模型和网络聚类,从科学论文中提取任务、机器学习方法和数据集名称,并分析这些信息之间的关系。所提出方法的表达式提取性能在使用Llama3时,在各个类别中均取得了超过0.8的F分数,证实了其实用性。在金融领域论文上的基准测试结果证明了该方法的有效性,为最新数据集(包括与ESG(环境、社会和治理)数据相关的数据集)的使用提供了见解。

英文摘要:

Machine learning is widely utilized across various industries. Identifying the appropriate machine learning models and datasets for specific tasks is crucial for the effective industrial application of machine learning. However, this requires expertise in both machine learning and the relevant domain, leading to a high learning cost. Therefore, research focused on extracting combinations of tasks, machine learning models, and datasets from academic papers is critically important, as it can facilitate the automatic recommendation of suitable methods. Conventional information extraction methods from academic papers have been limited to identifying machine learning models and other entities as named entities. To address this issue, this study proposes a methodology extracting tasks, machine learning methods, and dataset names from scientific papers and analyzing the relationships between these information by using LLM, embedding model, and network clustering. The proposed method's expression extraction performance, when using Llama3, achieves an F-score exceeding 0.8 across various categories, confirming its practical utility. Benchmarking results on financial domain papers have demonstrated the effectiveness of this method, providing insights into the use of the latest datasets, including those related to ESG (Environmental, Social, and Governance) data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑