arXiv 2203.080672022-03-17cs.LGcs.CRstat.APstat.CO

Practical data monitoring in the internet-services domain

Nikhil Galagali

英文摘要：

Large-scale monitoring, anomaly detection, and root cause analysis of metrics are essential requirements of the internet-services industry. To address the need to continuously monitor millions of metrics, many anomaly detection approaches are being used on a daily basis by large internet-based companies. However, in spite of the significant progress made to accurately and efficiently detect anomalies in metrics, the sheer scale of the number of metrics has meant there are still a large number of false alarms that need to be investigated. This paper presents a framework for reliable large-scale anomaly detection. It is significantly more accurate than existing approaches and allows for easy interpretation of models, thus enabling practical data monitoring in the internet-services domain.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

DRLD-SP: A Deep Reinforcement Learning-based Dynamic Service Placement in Edge-Enabled Internet of Vehicles

National University of Singapore（新加坡国立大学）

The growth of 5G and edge computing has enabled the emergence of Internet of Vehicles. It supports different types of services with different resource and service requirements. However, limited resources at the edge, high mobility of vehicles, increasing demand, and dynamicity in service request-types have made service placement a challenging task. A typical static placement solution is not effective as it does not consider the traffic mobility and service dynamics. Handling dynamics in IoV for service placement is an important and challenging problem which is the primary focus of our work in …

2021-09-08cs.LGcs.NI

Optimal Pricing of Internet of Things: A Machine Learning Approach

University of Canberra（堪培拉大学）；University of Technology Sydney（悉尼科技大学）；Nanyang Technological University（南洋理工大学）；York University（约克大学）；University of Houston（休斯顿大学）；Kyung Hee University（庆熙大学）

Internet of things (IoT) produces massive data from devices embedded with sensors. The IoT data allows creating profitable services using machine learning. However, previous research does not address the problem of optimal pricing and bundling of machine learning-based IoT services. In this paper, we define the data value and service quality from a machine learning perspective. We present an IoT market model which consists of data vendors selling data to service providers, and service providers offering IoT services to customers. Then, we introduce optimal pricing schemes for the standalone an…

2020-02-14cs.LGcs.NI

用于安全物联网设备监控服务的联邦卡尔曼滤波器

Federated Kalman Filter for Secure IoT-based Device Monitoring Services

University of Guelph（圭尔夫大学）

本文提出一种结合联邦卡尔曼滤波器、联邦学习与私有区块链的平台，用于解决物联网设备监控服务中的隐私问题，实验证明其在基于RSSI的定位数据估计上具有显著改进潜力。

2023-04-03cs.CRcs.LGcs.NI

Adaptive Learning for Service Monitoring Data

University of Regina（里贾纳大学）；Ericsson Canada Inc.（加拿大爱立信公司）

Service monitoring applications continuously produce data to monitor their availability. Hence, it is critical to classify incoming data in real-time and accurately. For this purpose, our study develops an adaptive classification approach using Learn++ that can handle evolving data distributions. This approach sequentially predicts and updates the monitoring model with new data, gradually forgets past knowledge and identifies sudden concept drift. We employ consecutive data chunks obtained from an industrial application to evaluate the performance of the predictors incrementally.

2022-08-25cs.LG

Predicting IPv4 Services Across All Ports

Stanford University（斯坦福大学）；Inria（法国国家信息与自动化研究所）

Internet-wide scanning is commonly used to understand the topology and security of the Internet. However, IPv4 Internet scans have been limited to scanning only a subset of services -- exhaustively scanning all IPv4 services is too costly and no existing bandwidth-saving frameworks are designed to scan IPv4 addresses across all ports. In this work we introduce GPS, a system that efficiently discovers Internet services across all ports. GPS runs a predictive framework that learns from extremely small sample sizes and is highly parallelizable, allowing it to quickly find patterns between service…

2023-03-02cs.DCcs.LGcs.NI

A Matrix Factorization Model for Hellinger-based Trust Management in Social Internet of Things

Sharif University of Technology（谢里夫理工大学）；Boston University（波士顿大学）；McMaster University（麦克马斯特大学）

The Social Internet of Things (SIoT), integration of the Internet of Things and Social Networks paradigms, has been introduced to build a network of smart nodes that are capable of establishing social links. In order to deal with misbehaving service provider nodes, service requestor nodes must evaluate their trustworthiness levels. In this paper, we propose a novel trust management mechanism in the SIoT to predict the most reliable service providers for each service requestor, which leads to reduce the risk of being exposed to malicious nodes. We model the SIoT with a flexible bipartite graph …

2021-03-02cs.CRcs.LGcs.SI

更多相关论文收起更多论文

Service Discovery in Social Internet of Things using Graph Neural Networks

King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）

Internet-of-Things (IoT) networks intelligently connect thousands of physical entities to provide various services for the community. It is witnessing an exponential expansion, which is complicating the process of discovering IoT devices existing in the network and requesting corresponding services from them. As the highly dynamic nature of the IoT environment hinders the use of traditional solutions of service discovery, we aim, in this paper, to address this issue by proposing a scalable resource allocation neural model adequate for heterogeneous large-scale IoT networks. We devise a Graph N…

2022-06-18cs.DCcs.LG

联邦物联网中基于非参数子空间监测的对抗客户端检测

Adversarial Client Detection via Non-parametric Subspace Monitoring in the Internet of Federated Things

Arizona State University（亚利桑那州立大学）；University of Florida（佛罗里达大学）；Clemson University（克莱姆森大学）

针对联邦物联网中联邦学习易遭对抗攻击的安全问题，提出非参数方法FedRR，利用传输参数更新的低秩特征检测对抗客户端，可在无攻击场景下控制误报率，经MNIST数字识别实验验证有效。

2023-10-02cs.LG

Energy-Efficient Real-Time Heart Monitoring on Edge-Fog-Cloud Internet-of-Medical-Things

University of California at Irvine（加州大学尔湾分校）

The recent developments in wearable devices and the Internet of Medical Things (IoMT) allow real-time monitoring and recording of electrocardiogram (ECG) signals. However, continuous monitoring of ECG signals is challenging in low-power wearable devices due to energy and memory constraints. Therefore, in this paper, we present a novel and energy-efficient methodology for continuously monitoring the heart for low-power wearable devices. The proposed methodology is composed of three different layers: 1) a Noise/Artifact detection layer to grade the quality of the ECG signals; 2) a Normal/Abnorma…

2021-12-15cs.LGcs.PFeess.SP

Federated Learning for Autoencoder-based Condition Monitoring in the Industrial Internet of Things

Technische Universität Berlin（柏林工业大学）

Enabled by the increasing availability of sensor data monitored from production machinery, condition monitoring and predictive maintenance methods are key pillars for an efficient and robust manufacturing production cycle in the Industrial Internet of Things. The employment of machine learning models to detect and predict deteriorating behavior by analyzing a variety of data collected across several industrial environments shows promising results in recent works, yet also often requires transferring the sensor data to centralized servers located in the cloud. Moreover, although collaborating a…

2022-11-14cs.LG

DiagNet: towards a generic, Internet-scale root cause analysis solution

InterDigital；Univ Rennes（雷恩大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；IRISA（IRISA 研究所）

Diagnosing problems in Internet-scale services remains particularly difficult and costly for both content providers and ISPs. Because the Internet is decentralized, the cause of such problems might lie anywhere between an end-user's device and the service datacenters. Further, the set of possible problems and causes is not known in advance, making it impossible in practice to train a classifier with all combinations of problems, causes and locations. In this paper, we explore how different machine learning techniques can be used for Internet-scale root cause analysis using measurements taken f…

2020-04-07cs.AIcs.LGcs.NE

面向云服务监控的自动化配置推荐综合框架

A Holistic Framework for Automated Configuration Recommendation for Cloud Service Monitoring

Microsoft India（微软印度）

本文提出一个模块化推荐框架，通过处理图结构服务实体来推荐最优监控配置，通过实验验证其在提供相关监控配置推荐方面的有效性。

2026-02-07cs.DCcs.LG