The rapid growth of emerging information technologies and application patterns in modern society, e.g., Internet, Internet of Things, Cloud Computing and Tri-network Convergence, has caused the advent of the era of big data. Big data contains huge values, however, mining knowledge from big data is a tremendously challenging task because of data uncertainty and inconsistency. Attribute reduction (also known as feature selection) can not only be used as an effective preprocessing step, but also exploits the data redundancy to reduce the uncertainty. However, existing solutions are designed 1) either for a single machine that means the entire data must fit in the main memory and the parallelism is limited; 2) or for the Hadoop platform which means that the data have to be loaded into the distributed memory frequently and therefore become inefficient. In this paper, we overcome these shortcomings for maximum efficiency possible, and propose a unified framework for Parallel Large-scale Attribute Reduction, termed PLAR, for big data analysis. PLAR consists of three components: 1) Granular Computing (GrC)-based initialization: it converts a decision table (i.e., original data representation) into a granularity representation which reduces the amount of space and hence can be easily cached in the distributed memory: 2) model-parallelism: it simultaneously evaluates all feature candidates and makes attribute reduction highly parallelizable; 3) data-parallelism: it computes the significance of an attribute in parallel using a MapReduce-style manner. We implement PLAR with four representative heuristic feature selection algorithms on Spark, and evaluate them on various huge datasets, including UCI and astronomical datasets, finding our method's advantages beyond existing solutions.
Identity-Aware Attribute Recognition via Real-Time Distributed Inference in Mobile Edge Clouds
The Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, School of Software, Dalian University of Technology(大连理工大学软件学院,辽宁省泛在网络与服务软件重点实验室);School of Software, Dalian University of Technology(大连理工大学软件学院);The Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, International School of Information Science and Engineering, Dalian University of Technology(大连理工大学国际信息与软件学院,辽宁省泛在网络与服务软件重点实验室);The Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology(华中科技大学网络空间安全学院,湖北省大数据安全工程研究中心)
Real-time and Downtime-tolerant Fault Diagnosis for Railway Turnout Machines (RTMs) Empowered with Cloud-Edge Pipeline Parallelism
School of Software, Nanjing University of Information Science and Technology(南京信息工程大学软件学院);School of Computing and Communications, Lancaster University(兰卡斯特大学计算与通信学院);NRIET Industrial Co.,Ltd.(国电南瑞科技股份有限公司)
Edge-assisted Parallel Uncertain Skyline Processing for Low-latency IoE Analysis
National Chung Cheng University(国立中正大学);Trend Micro Inc(趋势科技);Taiwan Semiconductor Manufacturing Company, Ltd (TSMC)(台湾积体电路制造股份有限公司);National Taipei University of Technology(台北科技大学)