请注意注意力：Transformer模型在流程预测中真正学习到了什么
Attention Please: What Transformer Models Really Learn for Process Prediction
AI总结：
本文研究Transformer下一活动预测模型的注意力分数能否解释决策，发现其可作为解释器并提出两种图基解释方法，为流程模型改进和事件日志挖掘提供启发。
摘要：
预测性流程监控旨在通过对流程实例的后续演化做出各类预测，为流程运行时的执行提供支持。近年来，大量深度学习架构已成为不同预测目标的当前最优方案，其中包括Transformer架构。Transformer架构配备了强大的注意力机制，为每个输入部分分配注意力分数，从而能够优先处理最相关的信息，生成更准确、更具上下文相关性的输出。然而，深度学习模型在很大程度上属于黑箱，即其推理或决策过程无法被详细理解。本文研究了基于Transformer的下一活动预测模型的注意力分数能否作为其决策的解释。我们发现，下一活动预测模型中的注意力分数可作为解释器，并基于这一发现提出了两种基于图的解释方法。所得见解可为未来预测性业务流程模型的改进研究提供启发，同时也为基于神经网络从事件日志中挖掘流程模型提供了可能。
英文摘要：
Predictive process monitoring aims to support the execution of a process during runtime with various predictions about the further evolution of a process instance. In the last years a plethora of deep learning architectures have been established as state-of-the-art for different prediction targets, among others the transformer architecture. The transformer architecture is equipped with a powerful attention mechanism, assigning attention scores to each input part that allows to prioritize most relevant information leading to more accurate and contextual output. However, deep learning models largely represent a black box, i.e., their reasoning or decision-making process cannot be understood in detail. This paper examines whether the attention scores of a transformer based next-activity prediction model can serve as an explanation for its decision-making. We find that attention scores in next-activity prediction models can serve as explainers and exploit this fact in two proposed graph-based explanation approaches. The gained insights could inspire future work on the improvement of predictive business process models as well as enabling a neural network based mining of process models from event logs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
