Hard Disk Drive (HDD) failures in datacenters are costly - from catastrophic data loss to a question of goodwill, stakeholders want to avoid it like the plague. An important tool in proactively monitoring against HDD failure is timely estimation of the Remaining Useful Life (RUL). To this end, the Self-Monitoring, Analysis and Reporting Technology employed within HDDs (S.M.A.R.T.) provide critical logs for long-term maintenance of the security and dependability of these essential data storage devices. Data-driven predictive models in the past have used these S.M.A.R.T. logs and CNN/RNN based architectures heavily. However, they have suffered significantly in providing a confidence interval around the predicted RUL values as well as in processing very long sequences of logs. In addition, some of these approaches, such as those based on LSTMs, are inherently slow to train and have tedious feature engineering overheads. To overcome these challenges, in this work we propose a novel transformer architecture - a Temporal-fusion Bi-encoder Self-attention Transformer (TFBEST) for predicting failures in hard-drives. It is an encoder-decoder based deep learning technique that enhances the context gained from understanding health statistics sequences and predicts a sequence of the number of days remaining before a disk potentially fails. In this paper, we also provide a novel confidence margin statistic that can help manufacturers replace a hard-drive within a time frame. Experiments on Seagate HDD data show that our method significantly outperforms the state-of-the-art RUL prediction methods during testing over the exhaustive 10-year data from Backblaze (2013-present). Although validated on HDD failure prediction, the TFBEST architecture is well-suited for other prognostics applications and may be adapted for allied regression problems.
Graph Neural Networks with Learnable Structural and Positional Representations
Nanyang Technological University(南洋理工大学);Loyola Marymount University(洛约拉马利蒙特大学);Mila(米拉);University of Montréal(蒙特利尔大学);CIFAR(加拿大高级研究所);National University of Singapore(新加坡国立大学)
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
Google Research(谷歌研究院);Columbia University(哥伦比亚大学);Carnegie Mellon University(卡内基梅隆大学);Waymo(Waymo公司);Peking University(北京大学);University of Cambridge(剑桥大学);Alan Turing Institute(阿兰·图灵研究所)
LEAP: Local ECT-Based Learnable Positional Encodings for Graphs
AIDOS Lab, University of Fribourg(弗里堡大学AIDOS实验室);Institute of AI for Health, Helmholtz Munich(慕尼黑亥姆霍兹人工智能与健康研究所);Technical University of Munich(慕尼黑技术大学);Department of Computer Science, ETH Zurich(苏黎世联邦理工学院计算机科学系);Department of Computer Science, University of Basel(巴塞尔大学计算机科学系)