听觉注意解码中性能指标为何会过度承诺：一种信息论的重新评估
Why Performance Metrics Overpromise in Auditory Attention Decoding: an Information-Theoretic Reappraisal
AI总结：
本文针对听觉注意解码（AAD）的性能指标问题，引入相对增量互信息（rIMI），发现直接分类型AAD算法并不优于传统刺激重构型算法，且其准确率存在人为虚高的问题。
AI中文摘要：
听觉注意解码（AAD）算法主要在受试者持续关注同一说话人的稳态下，采用准确率、信息传递率等指标进行评估。然而，这类指标未考虑AAD预测与先前预测之间的（非）依赖关系。本文指出，在算法评估中未考虑该依赖关系会严重扭曲AAD算法的真实性能，因此引入相对增量互信息（rIMI）——即新预测消除目标说话人身份剩余不确定性的速率，该指标可追踪预测在先前获取信息的基础上实际生成的新有用信息量。通过研究rIMI及AAD模型在注意力切换时的表现，我们发现，尽管仅从准确率来看，近期的直接分类型AAD算法似乎优于基于刺激重构的传统AAD算法，但实际并非如此；同时还发现，这些直接分类型AAD预测会受到无关特征漂移的严重影响，这种漂移会跨窗口甚至跨试次泄露信息，人为地抬高准确率。
英文摘要：
Auditory attention decoding (AAD) algorithms are predominantly evaluated in a steady state where a listener continuously attends to the same speaker, using metrics such as accuracy and information transfer rate. However, such metrics fail to account for the (in-)dependence of an AAD prediction with respect to previous predictions. In this paper, we argue that failing to take this dependence into account in the algorithm evaluation can lead to severe misrepresentations of the true performance of an AAD algorithm. We therefore introduce the relative Incremental Mutual Information (rIMI); the rate at which a new prediction removes the remaining uncertainty about the identity of the attended speaker. This allows us to track how much new, useful information a prediction actually generates on top of the information already obtained from previous predictions. By investigating the rIMI and the behaviour of AAD models around attention switches, we demonstrate that recent direct-classification AAD algorithms are not superior to traditional AAD algorithms based on stimulus reconstruction, despite what accuracy alone may suggest. We also demonstrate how these direct-classification AAD predictions are severely influenced by irrelevant feature drifts, which artificially inflates accuracies by leaking information across windows, and even across trials.
