The Sociolinguistic Foundations of Language Modeling
University of Birmingham（伯明翰大学）
Jack Grieve, Sara Bartl, Matteo Fuoli, Jason Grafmiller, Weihang Huang, Alejandro Jawerbaum, Akira Murakami, Marcus Perlman, Dana Roemling, Bodo Winter
In this paper, we introduce a sociolinguistic perspective on language modeling. We claim that large language models are inherently models of varieties of language, and we consider how this insight can inform the development and deployment of large language models. We begin by presenting a technical definition of the concept of a variety of language as developed in sociolinguistics. We then discuss how this perspective can help address five basic challenges in language modeling: social bias, domain adaptation, alignment, language change, and scale. Ultimately, we argue that it is crucial to carefully define and compile training corpora that accurately represent the specific varieties of language being modeled to maximize the performance and societal value of large language models.
TFD: A Comprehensive Structured Tibetan Foundation Dataset for Low-Resource Language Processing and Large-Scale Modeling
University of Electronic Science and Technology of China（电子科技大学）；Xizang University（西藏大学）；ZenWeave AI；The State Key Laboratory of Tibetan Intelligence（藏语智能国家重点实验室）；Nanyang Technological University（南洋理工大学）
Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学陈笃生公共卫生学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究院）
AntigenLM: Structure-Aware DNA Language Modeling for Influenza
Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；Beijing Institute of Genomics, Chinese Academy of Sciences（中国科学院北京基因组研究所）；China National Center for Bioinformation（中国生物信息学国家中心）；University of Chinese Academy of Sciences（中国科学院大学）
Unsupervised Discovery of Gendered Language through Latent-Variable Modeling
University of Maryland, College Park（马里兰大学帕克分校）；Johns Hopkins University（约翰斯·霍普金斯大学）；Microsoft Research（微软研究院）；University of Cambridge（剑桥大学）；University of Copenhagen（哥本哈根大学）
Multilingual Agent-Based World Modeling for Social Science
Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；Singapore University of Technology and Design（新加坡科技设计大学）；National School of Development, Peking University（北京大学国家发展研究院）；Singapore Management University（新加坡管理大学）
