In this paper, we introduce a reproducible cleaning process for the text extracted from PDFs using n-gram models. Our approach compares the originally extracted text with the text generated from, or expected by, these models using earlier text as stimulus. To guide this process, we introduce the notion of a consistency score, which refers to the proportion of text that is expected by the model. This is used to monitor changes during the cleaning process, and across different corpuses. We illustrate our process on text from the book Jane Eyre and introduce both a Shiny application and an R package to make our process easier for others to adopt.
Score Before You Speak: Improving Persona Consistency in Dialogue Generation using Response Quality Scores
School of Computer Science, University of Leeds(计算机科学学院,利兹大学);Leeds Institute of Medical Education, School of Medicine, University of Leeds(利兹医学教育研究所,医学院,利兹大学)
Can We Trust LLMs for Mental Health Screening? Consistency, ASR Robustness, and Evidence Faithfulness
Usher Institute, Edinburgh Medical School, University of Edinburgh, Edinburgh, UK(埃瑟尔研究所、爱丁堡医学院、爱丁堡大学、爱丁堡);Department of Computer Engineering, Ahvaz Campus, Islamic Azad University, Ahvaz, Iran(计算机工程系、阿瓦兹校区、伊斯兰Azad大学、阿瓦兹,伊朗);School of Health and Social Care, Edinburgh Napier University, Edinburgh, UK(健康与社会护理学院、爱丁堡纳皮尔大学、爱丁堡,英国)
TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyML
Multidisciplinary Faculty of Nador University Mohammed Premier Oujda(穆罕默德·布雷德·奥杰达大学多学科学院);Laboratory LaSTI, ENSAK Sultan Moulay Slimane University Khouribga(库尔希格大学拉斯特伊实验室)