We introduce LM-Lexicon, an innovative definition modeling approach that incorporates data clustering, semantic expert learning, and model merging using a sparse mixture-of-experts architecture. By decomposing the definition modeling task into specialized semantic domains, where small language models are trained as domain experts, LM-Lexicon achieves substantial improvements (+7% BLEU score compared with the prior state-of-the-art model) over existing methods on five widely used benchmarks. Empirically, we demonstrate that 1) the clustering strategy enables fine-grained expert specialization with nearly 10% improvement in definition quality; 2) the semantic-aware domain-level routing mechanism achieves higher expert efficacy (+1%) than conventional token-level routing; and 3) further performance gains can be obtained through test-time compute and semantic expert scaling. Our work advances definition modeling while providing insights into the development of efficient language models for semantic-intensive applications.
CARE-SD: Classifier-based analysis for recognizing and eliminating stigmatizing and doubt marker labels in electronic health records: model development and validation
Emory University(埃默里大学);University of Florida(佛罗里达大学);Children’s Healthcare of Atlanta(亚特兰大儿童医疗中心);Mount Sinai(西奈山医院);Georgia Institute of Technology(佐治亚理工学院)
The Promise of Large Language Models in Digital Health: Evidence from Sentiment Analysis in Online Health Communities
School of Business and Management, Queen Mary University of London(女王玛丽大学商学院);Wolfson Institute of Population Health (WIPH), Queen Mary University of London(人口健康沃尔夫森研究所);Department of Medicine, University of Nottingham(诺丁汉大学医学部);Department of Informatics, Systems, and Communication, University of Milano-Bicocca(米兰-比科卡大学信息学、系统与通信系)
HYBRINFOX at CheckThat! 2024 -- Task 2: Enriching BERT Models with the Expert System VAGO for Subjectivity Detection
Université de Rennes(雷恩大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);IRISA(雷恩信息、信号与图像研究所);LIP6(巴黎第六大学信息学实验室);Sorbonne Université(索邦大学);Institut Jean-Nicod(让·尼科研究所);ENS-PSL(巴黎高等师范学院-巴黎文理研究大学);EHESS(法国高等社会科学研究院)
From Large Language Models to Knowledge Graphs for Biomarker Discovery in Cancer
Fraunhofer FIT(弗劳恩霍夫应用研究促进协会信息技术研究所);University of Siegen(锡根大学);University of Cologne(科隆大学);University Hospital Cologne(科隆大学医院);ZBMED - Information Center for Life Sciences(ZBMED生命科学信息中心);RWTH Aachen University(亚琛工业大学)