一种基于子词嵌入的方法用于卢森堡语用户评论中的变体检测
A Subword Embedding Approach for Variation Detection in Luxembourgish User Comments
AI总结：
本文提出一种基于子词嵌入的方法，用于检测卢森堡语用户评论中的词汇和正字法变异，通过余弦相似度和n-gram相似度实现，揭示了方言和社会语言学研究中的模式。
摘要：
本文提出了一种基于嵌入的方法，用于在不依赖先前规范化或预定义变体列表的情况下检测变体。该方法在原始文本上训练子词嵌入，并通过结合余弦相似度和n-gram相似度将相关形式分组。这使得拼写和形态多样性可以作为语言结构进行考察和分析，而不是被视为噪声。使用大量卢森堡语用户评论语料库，该方法揭示了广泛的词汇和正字法变异，这些变异与方言和社会语言学研究中描述的模式一致。诱导出的家族捕捉到了系统性的对应关系，并突显了地区性和风格性的差异。该过程不严格需要人工标注，但会产生透明的聚类，支持定量和定性分析。结果表明，分布建模可以在''嘈杂''或低资源环境下揭示有意义的变异模式，提供了一种可重复的方法论框架，用于研究多语言和小语言中的语言变体。
英文摘要：
This paper presents an embedding-based approach to detecting variation without relying on prior normalisation or predefined variant lists. The method trains subword embeddings on raw text and groups related forms through combined cosine and n-gram similarity. This allows spelling and morphological diversity to be examined and analysed as linguistic structure rather than treated as noise. Using a large corpus of Luxembourgish user comments, the approach uncovers extensive lexical and orthographic variation that aligns with patterns described in dialectal and sociolinguistic research. The induced families capture systematic correspondences and highlight areas of regional and stylistic differentiation. The procedure does not strictly require manual annotation, but does produce transparent clusters that support both quantitative and qualitative analysis. The results demonstrate that distributional modelling can reveal meaningful patterns of variation even in ''noisy'' or low-resource settings, offering a reproducible methodological framework for studying language variety in multilingual and small-language contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
