NEST：用于多重集序列的嵌套事件流变压器
NEST: Nested Event Stream Transformer for Sequences of Multisets
AI总结：
本文提出NEST，一种用于多重集序列事件流的模型，通过保留层次结构提升计算效率和表示质量，实验显示其在预训练效率和下游任务表现上均有提升。
摘要：
事件流数据往往具有层次结构，其中多个事件同时发生，形成多重集序列。在电子健康记录中，医疗事件被分组为具有明确时间结构的临床会诊序列，但每个会诊内事件的顺序和时间可能不确定或不可靠。现有事件流数据基础模型将这种层次结构扁平化为一维序列，导致计算效率低下和表示质量差。本文表明，在基础模型架构中保留原始层次结构能提供有用的归纳偏差，提升计算效率和表示质量。我们引入了嵌套事件流变压器（NEST），一种处理多重集序列事件流的基础模型。基于此架构，我们提出了掩码集合建模（MSM）高效范式，促进改进的集合级表示学习。实验表明，NEST在真实世界多重集序列数据上能捕捉真实动态，同时提升预训练效率和下游任务表现。
英文摘要：
Event stream data often exhibit hierarchical structure in which multiple events co-occur, resulting in a sequence of multisets (i.e., bags of events). In electronic health records (EHRs), for example, medical events are grouped into a sequence of clinical encounters with well-defined temporal structure, but the order and timing of events within each encounter may be unknown or unreliable. Most existing foundation models (FMs) for event stream data flatten this hierarchy into a one-dimensional sequence, leading to (i) computational inefficiency associated with dense attention and learning spurious within-set relationships, and (ii) lower-quality set-level representations from heuristic post-training pooling for downstream tasks. Here, we show that preserving the original hierarchy in the FM architecture provides a useful inductive bias that improves both computational efficiency and representation quality. We then introduce Nested Event Stream Transformer (NEST), a FM for event streams comprised of sequences of multisets. Building on this architecture, we formulate Masked Set Modeling (MSM), an efficient paradigm that promotes improved set-level representation learning. Experiments on real-world multiset sequence data show that NEST captures real-world dynamics while improving both pretraining efficiency and downstream performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
