Of the over 7,000 languages spoken in the world, commercial language identification (LID) systems only reliably identify a few hundred in written form. Research-grade systems extend this coverage under certain circumstances, but for most languages coverage remains patchy or nonexistent. This position paper argues that this situation is largely self-imposed. In particular, it arises from a persistent framing of LID as decontextualized text classification, which obscures the central role of prior probability estimation and is reinforced by institutional incentives that favor global, fixed-prior models. We argue that improving coverage for tail languages requires rethinking LID as a routing problem and developing principled ways to incorporate environmental cues that make languages locally plausible.
An Unsupervised Probability Model for Speech-to-Translation Alignment of Low-Resource Languages
University of Notre Dame(圣母大学);University of Melbourne(墨尔本大学)
对于许多低资源的语言,口语资源更有可能用翻译而不是translation来注释。翻译后的语音数据对于记录濒危语言或训练语音翻译系统具有潜在的价值。利用这些数据的第一步是自动将口语与其翻译对齐。我们提出了一个模型,结合戴尔等人。的IBM Model 2(快速对齐)和k-means聚类的重新参数化,使用Dynamic Time Warping作为距离度量。这两个组件使用期望最大化联合训练。在资源极其匮乏的情况下,我们的模型的表现明显优于神经模型和强基线。
Probability Distributions Computed by Autoregressive Transformers
University of Notre Dame(内布拉斯加大学达灵顿分校);ETH Zürich(苏黎世联邦理工学院);Max-Planck Institute for Software Systems(马克斯·普朗克软件系统研究所);University of Kaiserslautern-Landau(凯撒斯劳滕-劳恩堡大学);San José State University(圣何塞州立大学)
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);Center for Data Science, Peking University(北京大学数据科学中心);The MOE Key Laboratory of Computational Linguistics, Peking University(北京大学计算语言学教育部重点实验室)
Transcormer: Transformer for Sentence Scoring with Sliding Language Modeling
Microsoft Research Asia(微软亚洲研究院);Fudan University(复旦大学)
句子评分旨在测量句子的似然度得分,并广泛用于许多自然语言处理场景,如重新排序,即从多个候选中选择最佳句子。以往的句子评分方法主要采用GPT等因果语言模型(CLM)或BERT等掩蔽语言模型(MLM),但都存在一定的局限性:1)CLM只利用单向信息进行句子概率估计,没有考虑双向上下文,影响评分质量; 2)MLM一次只能估计部分标记的概率,因此需要多次向前传递来估计整个句子的概率,这会导致大量的计算和时间成本。在本文中,我们提出了\textit{Transcormer} -一个Transformer模型与一个新的\textit{sliding language modeling}(SLM)的句子评分。具体来说,我们的SLM采用了三重流自我注意力机制,以估计所有的令牌在一个句子中的概率与双向上下文,只需要一个单一的向前通过。SLM方法可以避免CLM(单向上下文)和MLM(多个前向传递)的局限性,并继承它们的优点,从而实现高效的评分。多个任务的实验结果表明,我们的方法取得了更好的性能比其他语言建模。
Position Paper On Diagnostic Uncertainty Estimation from Large Language Models: Next-Word Probability Is Not Pre-test Probability
University of Colorado Anschutz(科罗拉多大学安舒茨分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Harvard Medical School(哈佛医学院);Loyola University Chicago(芝加哥洛约拉大学);Boston Childrens Hospital(波士顿儿童医院);Dana Farber Cancer Institute(丹娜法伯癌症研究院)
Probability Consistency in Large Language Models: Theoretical Foundations Meet Empirical Discrepancies
EmpiriQal Inc.(EmpiriQal公司);Department of Mathematics London School of Economics and Political Science(伦敦政治经济学院数学系);Department of Psychology University of Tübingen(图宾根大学心理学系);Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室)
ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Models
School of Informatics, Xiamen University, Xiamen, China(信息学院,厦门大学,厦门);School of Film, Xiamen University, Xiamen, China(电影学院,厦门大学,厦门);Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism, Xiamen University, Xiamen, China(福建省和台湾非物质文化遗产数字保护与智能处理重点实验室,文化和旅游部,厦门大学,厦门);Institute of Artificial Intelligence, Xiamen University, Xiamen, China(人工智能学院,厦门大学,厦门);School of Computer and Data Science, Minjiang University, Fuzhou, China(计算机与数据科学学院,闽江大学,福州)
The potential of large language models for improving probability learning: A study on ChatGPT3.5 and first-year computer engineering students
Rey Juan Carlos University(胡安·卡洛斯国王大学);European Commission, Joint Research Centre(欧盟委员会联合研究中心);CETINIA: Research Centre for Intelligent Information Technologies(CETINIA智能信息技术研究中心)
Language Models can Evaluate Themselves via Probability Discrepancy
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Alibaba Group(阿里巴巴集团);International Center of Future Science, Jilin University(吉林大学未来科学国际中心)
Probabilistic Constraint Logic Programming. Formal Foundations of Quantitative and Statistical Inference in Constraint-Based Natural Language Processing
Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
Department of Brain and Cognitive Sciences, MIT(麻省理工学院脑科学与认知科学系);MIT Libraries CREOS(麻省理工学院图书馆 CREOS);Deparmtent of Cognitive Science, UCSD(加州大学圣地亚哥分校认知科学系)
SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models
Northeastern University(东北大学);CAS Key Laboratory of Behavioral Science(中国科学院行为科学重点实验室);Independent Researcher(独立研究员);Kunming University of Science and Technology(昆明理工大学)