arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.05378cs.CLcs.SI
收藏

超越注意力范式:从地理空间社交媒体数据中进行表示学习

Transcending the Attention Paradigm: Representation Learning from Geospatial Social Media Data

California Baptist University(加州浸会大学)

Nick DiSanto, Anthony Corso, Benjamin Sanders, Gavin Harding

AI总结:

本文通过分析超过20亿条地理标记推文并构建城市级词袋嵌入,证明地理位置显著影响在线交流,揭示无需复杂模型即可从嘈杂社交媒体数据中发现潜在模式,挑战了注意力范式和复杂模型在自然语言模式识别中的必要性。

摘要:

虽然Transformer率先提出了以注意力驱动的架构,并将其作为语言建模的基石,但它们对显式上下文信息的依赖凸显了其在隐性学习总体文本主题能力上的局限。本研究通过将社交媒体数据作为分布式模式的来源,挑战了性能基准测试的启发式范式。与依赖捕捉复杂长期依赖关系的网络形成鲜明对比的是,在线数据模型本质上缺乏结构,并被迫在聚合中检测潜在结构。为了恰当地表示这些抽象关系,本研究将经验性社交媒体语料库分解为其基本组成部分,分析了人口密集地区超过20亿条推文。我们为每个城市创建了特定的词袋嵌入,并比较了它们各自的表示。结果发现,即使在嘈杂的数据中,地理位置对在线交流也有相当大的影响,并且无需借助先进算法也能揭示隐藏的见解。这一证据为社会科学提供了有价值的地理空间启示,并挑战了复杂模型是自然语言模式识别先决条件的观念。这与不断演变的格局相一致,即质疑对绝对可解释性的推崇胜过抽象理解,并弥合了复杂框架与无形关系之间的鸿沟。

英文摘要:

While transformers have pioneered attention-driven architectures as a cornerstone of language modeling, their dependence on explicitly contextual information underscores limitations in their abilities to tacitly learn overarching textual themes. This study challenges the heuristic paradigm of performance benchmarking by investigating social media data as a source of distributed patterns. In stark contrast to networks that rely on capturing complex long-term dependencies, models of online data inherently lack structure and are forced to detect latent structures in the aggregate. To properly represent these abstract relationships, this research dissects empirical social media corpora into their elemental components, analyzing over two billion tweets across population-dense locations. We create Bag-of-Word embedding specific to each city and compare their respective representations. This finds that even amidst noisy data, geographic location has a considerable influence on online communication, and that hidden insights can be uncovered without the crutch of advanced algorithms. This evidence presents valuable geospatial implications in social science and challenges the notion that intricate models are prerequisites for pattern recognition in natural language. This aligns with the evolving landscape that questions the embrace of absolute interpretability over abstract understanding and bridges the divide between sophisticated frameworks and intangible relationships.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑