Due to the black box nature of deep neural networks (DNN), the continuous validation of DNN during operation is challenging with the absence of a human monitor. As a result this makes it difficult for developers and regulators to gain confidence in the deployment of autonomous systems employing DNN. It is critical for safety during operation to know when DNN's predictions are trustworthy or suspicious. With the absence of a human monitor, the basic approach is to use the model's output confidence score to assess if predictions are trustworthy or suspicious. However, the model's confidence score is a result of computations coming from a black box, therefore lacks transparency and makes it challenging to automatedly credit trustworthiness to predictions. We introduce the trustworthiness score (TS), a simple metric that provides a more transparent and effective way of providing confidence in DNN predictions compared to model's confidence score. The metric quantifies the trustworthiness in a prediction by checking for the existence of certain features in the predictions made by the DNN. We also use the underlying idea of the TS metric, to provide a suspiciousness score (SS) in the overall input frame to help in the detection of suspicious frames where false negatives exist. We conduct a case study using YOLOv5 on persons detection to demonstrate our method and usage of TS and SS. The case study shows that using our method consistently improves the precision of predictions compared to relying on model confidence score alone, for both 1) approving of trustworthy predictions (~20% improvement) and 2) detecting suspicious frames (~5% improvement).
Trustworthy DNN Partition for Blockchain-enabled Digital Twin in Wireless IIoT Networks
School of Electronic and Optical Engineering, Nanjing University of Science and Technology(南京理工大学电子与光学工程学院);Department of Computing, Hong Kong Polytechnic University(香港理工大学计算机系);Data61, CSIRO(数据61,CSIRO);Department of Electronics Engineering, Shanghai Jiao Tong University(上海交通大学电子工程系);National Mobile Communications Research Laboratory, Southeast University(东南大学国家移动通信研究中心)
DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score
NTT Media Intelligence Laboratories, NTT Corporation(NTT(日本电报电话公司)媒体情报实验室);University of Auckland(奥克兰大学);The University of Electro-Communications(电气通信大学)
Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores
University of Toronto(多伦多大学);University of Tennessee(田纳西大学);Vector Institute for Artificial Intelligence(人工智能矢量研究所);Schwartz Reisman Institute for Technology and Society(技术与社会斯瓦茨-雷曼研究所)
Enhancing Mutual Trustworthiness in Federated Learning for Data-Rich Smart Cities
Mohammad Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Polytechnique Montréal(蒙特利尔理工学院);Lebanese American University(黎巴嫩美国大学);Khalifa University(哈利法大学);New York University(纽约大学);Technology Innovation Institute (TII)(技术创新研究所)
ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction
Massachusetts Institute of Technology(麻省理工学院);University of California, San Diego(加州大学圣地亚哥分校);MIT-IBM Watson AI Lab(麻省理工-IBM沃森人工智能实验室);IBM Research(IBM研究院);Thomas J. Watson Research Center(托马斯·J·沃森研究中心)
Better Uncertainty Calibration via Proper Scores for Classification and Beyond
German Cancer Research Center (DKFZ)(德国癌症研究中心);German Cancer Consortium (DKTK)(德国癌症联盟);Goethe University Frankfurt(法兰克福大学);Frankfurt Cancer Institute(法兰克福癌症研究所)
An Explainable Adversarial Robustness Metric for Deep Learning Neural Networks
Department of Electrical and Computer Engineering, University of Illinois at Chicago(伊利诺伊大学芝加哥分校电气与计算机工程系);Kitware Inc.(Kitware公司);University of Illinois at Chicago(伊利诺伊大学芝加哥分校)