Metadata play a crucial role in ensuring the findability, accessibility, interoperability, and reusability of datasets. This paper investigates the potential of large language models (LLMs), specifically GPT-4, to improve adherence to metadata standards. We conducted experiments on 200 random data records describing human samples relating to lung cancer from the NCBI BioSample repository, evaluating GPT-4's ability to suggest edits for adherence to metadata standards. We computed the adherence accuracy of field name-field value pairs through a peer review process, and we observed a marginal average improvement in adherence to the standard data dictionary from 79% to 80% (p<0.5). We then prompted GPT-4 with domain information in the form of the textual descriptions of CEDAR templates and recorded a significant improvement to 97% from 79% (p<0.01). These results indicate that, while LLMs may not be able to correct legacy metadata to ensure satisfactory adherence to standards when unaided, they do show promise for use in automated metadata curation when integrated with a structured knowledge base
Self-Describing Structured Data with Dual-Layer Guidance: A Lightweight Alternative to RAG for Precision Retrieval in Large-Scale LLM Knowledge Navigation
Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness?
AstraZeneca(阿斯利康);Uppsala University(乌普萨拉大学);KTH Royal Institute of Technology(瑞典皇家理工学院);Science for Life Laboratory(生命科学实验室);Chalmers University of Technology(查尔姆斯理工大学);University of Gothenburg(哥德堡大学)
AMAQA: A Metadata-based QA Dataset for RAG Systems
Institute for Informatics and Telematics, National Research Council(信息与电信研究院,国家研究院);Department of Information Engineering, University of Pisa(信息工程系,比萨大学);Department of Computer Science, University of Pisa(计算机科学系,比萨大学)
Istituto per le Applicazioni del Calcolo Mauro Picone, Italian National Research Council(意大利国家研究委员会 Mauro Picone 应用计算研究所);Università Campus Bio-Medico di Roma(罗马校园生物医学大学);UniCamillus-Saint Camillus International University of Health Sciences(圣卡米勒斯国际健康科学大学)
CurateGPT: A flexible language-model assisted biocuration tool
Lawrence Berkeley National Laboratory(劳伦斯伯克利国家实验室);Queen Mary University of London(伦敦玛丽女王大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);European Bioinformatics Institute (EMBL-EBI)(欧洲生物信息研究所(欧洲分子生物学实验室-欧洲生物信息研究所));Berlin Institute of Health - Charité, Universitätsmedizin(柏林健康研究所-夏里特医科大学)
RedPajama: an Open Dataset for Training Large Language Models
Together AI;Stanford University(斯坦福大学);University of Chicago(芝加哥大学);EleutherAI;Princeton University(普林斯顿大学);ETH Zurich(苏黎世联邦理工学院);Mila(米拉);Université de Montréal(蒙特利尔大学);Ohio State University(俄亥俄州立大学);Caltech(加州理工学院)
Identity resolution of software metadata using Large Language Models
Barcelona Supercomputing Center, Barcelona, Spain(巴塞罗那超级计算中心);Department of Biochemistry and Molecular Biology, University of Barcelona, Barcelona, Spain(生物化学与分子生物学系,巴塞罗那大学)
Evaluating Large Language Models for Multimodal Simulated Ophthalmic Decision-Making in Diabetic Retinopathy and Glaucoma Screening
Ophthalmology Department, São Paulo Federal University(圣保罗联邦大学眼科部);Université Paris-Saclay, CentraleSupélec, Mathematics and Computer Science Laboratory(巴黎萨克莱大学、中央超导实验室数学与计算机科学实验室);Laboratory for Computational Physiology, Massachusetts Institute of Technology(计算生理学实验室、麻省理工学院)
Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools
Cyberspace Institute of Advanced Technology, Guangzhou University(广州大学网络空间研究院);Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(香港理工大学电子与电气工程系)
FlyAOC: Evaluating Agentic Ontology Curation of Drosophila Scientific Knowledge Bases
University of Michigan(密歇根大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学)
Curation of a Palaeohispanic Dataset for Machine Learning
Department of Computer Science and Artificial Intelligence, Universidad de Sevilla(塞维利亚大学计算机科学与人工智能系);Department of Spanish Language, Linguistics and Theory of Literature, University of Seville(塞维利亚大学西班牙语言、语言学与文学理论系)
FOSSIL: Regret-Minimizing Curriculum Learning for Metadata-Free and Low-Data Mpox Diagnosis
Computer Science Division, Gwinnett Technical College, Lawrenceville, GA, USA(基普顿技术学院计算机科学系);Department of Computer Science, University of Kentucky, Lexington, KY, USA(肯塔基大学计算机科学系);Department of Biomedical Engineering, Kumoh National Institute of Technology, Gumi, South Korea(Kumoh国立技术学院生物医学工程系);Department of Biology, Georgia Institute of Technology, Atlanta, GA, USA(佐治亚理工学院生物学系);Oaro Dermatology Center, Seongnam, Gyeonggi-do, South Korea(Oaro皮肤科中心)
Acquisition state behaves as a structured, measurable variable governing lung-nodule AI: kernel-driven measurement instability and noise-driven detection fragility, invisible to DICOM metadata