发表机构
Sber; Sber AI Lab（俄罗斯储蓄银行; 俄罗斯储蓄银行人工智能实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究金融应用中多模态事件序列预测建模，提出预训练基础变压器模型统一多源事件成序列，经下一个事件预测学习通用表示，结合现有特征训练轻量级神经模型用于多下游任务，优于传统模型并减少开发开销且已应用取得业务改进。
AI 中文摘要
预测建模是现代金融服务的核心组成部分，传统上使用基于手动设计的表格特征训练的单独模型来处理各种任务。这种特定任务的方法限制了重用，并且难以充分利用诸如交易历史和数字交互信号等异构数据源。在本文中，我们提出了一种基于在用户事件的多模态序列上预训练基础变压器模型的方法。来自多个数据源的事件被统一到一个单一的时间序列中，通过下一个事件预测目标实现异构模态的早期融合和通用表示的学习。这些表示与现有的工程用户特征相结合，在此基础上训练轻量级神经模型用于多个下游任务。所提出的系统优于传统的特定任务模型，同时减少了开发开销。该方法已在东欧最大的银行之一投入生产，在业务指标上取得了可衡量的改进。
英文摘要
Predictive modeling is a core component of modern financial services, where a wide range of tasks are traditionally addressed using separate models trained on manually engineered tabular features. This task-specific approach limits reuse and makes it difficult to fully exploit heterogeneous data sources such as transaction histories and digital interaction signals. In this paper, we present an approach based on pretraining a foundation transformer model on multimodal sequences of user events. Events from multiple data sources are unified into a single chronological sequence, enabling early fusion of heterogeneous modalities and learning of general-purpose representations via a next-event prediction objective. These representations are combined with existing engineered user features, on top of which lightweight neural models are trained for multiple downstream tasks. The proposed system outperforms traditional task-specific models while reducing development overhead. The approach was deployed in production at one of the biggest banks in Eastern Europe, resulting in measurable improvements in business metrics.
