Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.
University of Chinese Academy of Sciences（中国科学院大学）；Baidu Inc.（百度公司）；Rutgers, the State University of New Jersey（罗格斯新泽西州立大学）
Transfer learning aims at improving the performance of target learners on target domains by transferring the knowledge contained in different but related source domains. In this way, the dependence on a large number of target domain data can be reduced for constructing target learners. Due to the wide application prospects, transfer learning has become a popular and promising area in machine learning. Although there are already some valuable and impressive surveys on transfer learning, these surveys introduce approaches in a relatively isolated way and lack the recent advances in transfer lear…
Generating a description of an image is called image captioning. Image captioning requires to recognize the important objects, their attributes and their relationships in an image. It also needs to generate syntactically and semantically correct sentences. Deep learning-based techniques are capable of handling the complexities and challenges of image captioning. In this survey paper, we aim to present a comprehensive review of existing deep learning-based image captioning techniques. We discuss the foundation of the techniques to analyze their performances, strengths and limitations. We also d…
Northwest University（西北大学）；Monash University（莫纳什大学）；Carnegie Mellon University（卡内基梅隆大学）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））
Deep learning has made breakthroughs and substantial in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers' prior knowledge and experience. And due to the limitations of human' inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as m…
New Mexico Institute of mining and Technology（新墨西哥矿业技术学院）；Illinois State University（伊利诺伊州立大学）
Modern applications are highly sensitive to communication delays and throughput. This paper surveys major attempts on reducing latency and increasing the throughput. These methods are surveyed on different networks and surroundings such as wired networks, wireless networks, application layer transport control, Remote Direct Memory Access, and machine learning based transport control.
National University of Singapore（新加坡国立大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Deep learning have achieved promising results on a wide spectrum of AI applications. Larger datasets and models consistently yield better performance. However, we generally spend longer training time on more computation and communication. In this survey, we aim to provide a clear sketch about the optimizations for large-scale deep learning with regard to the model accuracy and model efficiency. We investigate algorithms that are most commonly used for optimizing, elaborate the debatable topic of generalization gap arises in large-batch training, and review the SOTA strategies in addressing the…
Western Michigan University（西密歇根大学）；Information Technology University（信息技术大学）
Future buildings will offer new convenience, comfort, and efficiency possibilities to their residents. Changes will occur to the way people live as technology involves into people's lives and information processing is fully integrated into their daily living activities and objects. The future expectation of smart buildings includes making the residents' experience as easy and comfortable as possible. The massive streaming data generated and captured by smart building appliances and devices contains valuable information that needs to be mined to facilitate timely actions and better decision mak…
Vellore Institute Of Technology（韦洛尔理工大学）；School of Computer Science and Engineering- SCOPE（计算机科学与工程学院-SCOPE）
Aspect Based Sentiment Analysis (ABSA) is the sub-field of Natural Language Processing that deals with essentially splitting our data into aspects ad finally extracting the sentiment information. ABSA is known to provide more information about the context than general sentiment analysis. In this study, our aim is to explore the various methodologies practiced while performing ABSA, and providing a comparative study. This survey paper discusses various solutions in-depth and gives a comparison between them. And is conveniently divided into sections to get a holistic view on the process.
University of Electronic Science and Technology of China（电子科技大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Lehigh University（理海大学）
Cluster analysis plays an indispensable role in machine learning and data mining. Learning a good data representation is crucial for clustering algorithms. Recently, deep clustering, which can learn clustering-friendly representations using deep neural networks, has been broadly applied in a wide range of clustering tasks. Existing surveys for deep clustering mainly focus on the single-view fields and the network architectures, ignoring the complex application scenarios of clustering. To address this issue, in this paper we provide a comprehensive survey for deep clustering in views of data so…
Machine Learning Innovations in CPR: A Comprehensive Survey on Enhanced Resuscitation Techniques
Concordia University（康考迪亚大学）；Aqaba University of Technology（亚喀巴科技大学）；Khalifa University（哈利法大学）；University of Alberta（阿尔伯塔大学）；Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）；Istinye University（伊斯蒂涅大学）；University of Waterloo（滑铁卢大学）
A Comprehensive Survey on Spectral Clustering with Graph Structure Learning
Queensland University of Technology (QUT)（昆士兰科技大学）；Graduate University of Advanced Technology（高等技术研究生院）；Ardakan University（阿尔达坎大学）；RMIT University（皇家墨尔本理工大学）
Amazon（亚马逊）；The Ohio State University（俄亥俄州立大学）；École de Technologie Supérieure（高等技术学院）；University of Minnesota（明尼苏达大学）
Collaborative recommendation approaches based on nearest-neighbors are still highly popular today due to their simplicity, their efficiency, and their ability to produce accurate and personalized recommendations. This chapter offers a comprehensive survey of neighborhood-based methods for the item recommendation problem. It presents the main characteristics and benefits of such methods, describes key design choices for implementing a neighborhood-based recommender system, and gives practical information on how to make these choices. A broad range of methods is covered in the chapter, including…
Lenovo Machine Intelligence Center（联想机器智能中心）；Delft University of Technology（代尔夫特理工大学）；The Chinese University of Hong Kong（香港中文大学）
Meta-learning has gained wide popularity as a training framework that is more data-efficient than traditional machine learning methods. However, its generalization ability in complex task distributions, such as multimodal tasks, has not been thoroughly studied. Recently, some studies on multimodality-based meta-learning have emerged. This survey provides a comprehensive overview of the multimodality-based meta-learning landscape in terms of the methodologies and applications. We first formalize the definition of meta-learning in multimodality, along with the research challenges in this growing…
The 2030 Agenda and accompanying Sustainable Development Goals (SDGs) are vital in guiding national and global policy. However, many of the SDG indicators used to measure progress toward those goals suffer from long publication lags. Nowcasting has the potential to address this problem and generate more timely estimates of those indicators. This paper provides resources for achieving that potential by 1) carrying out a comprehensive nowcasting feasibility survey of all SDG indicators to assess their potential to be nowcast, and 2) performing a case study of indicator 9.4.1 to illustrate and sh…
University of Kaiserslautern（凯泽斯劳滕大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
The emerging technology of multi-tenancy network slicing is considered as an essential feature of 5G cellular networks. It provides network slices as a new type of public cloud services, and therewith increases the service flexibility and enhances the network resource efficiency. Meanwhile, it raises new challenges of network resource management. A number of various methods have been proposed over the recent past years, in which machine learning and artificial intelligence techniques are widely deployed. In this article, we provide a survey to existing approaches of network slicing resource ma…
The difficulty of deploying various deep learning (DL) models on diverse DL hardware has boosted the research and development of DL compilers in the community. Several DL compilers have been proposed from both industry and academia such as Tensorflow XLA and TVM. Similarly, the DL compilers take the DL models described in different DL frameworks as input, and then generate optimized codes for diverse DL hardware as output. However, none of the existing survey has analyzed the unique design architecture of the DL compilers comprehensively. In this paper, we perform a comprehensive survey of exi…
Northeastern University（东北大学）；HKUST（香港科技大学）；Hong Kong Baptist University（香港浸会大学）
Federated learning utilizes various resources provided by participants to collaboratively train a global model, which potentially address the data privacy issue of machine learning. In such promising paradigm, the performance will be deteriorated without sufficient training data and other resources in the learning process. Thus, it is quite crucial to inspire more participants to contribute their valuable resources with some payments for federated learning. In this paper, we present a comprehensive survey of incentive schemes for federate learning. Specifically, we identify the incentive probl…
Deep neural networks are at the forefront of machine learning research. However, despite achieving impressive performance on complex tasks, they can be very sensitive: Small perturbations of inputs can be sufficient to induce incorrect behavior. Such perturbations, called adversarial examples, are intentionally designed to test the network's sensitivity to distribution drifts. Given their surprisingly small size, a wide body of literature conjectures on their existence and how this phenomenon can be mitigated. In this article we discuss the impact of adversarial examples on security, safety, a…
Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC ev…
Southeast University（东南大学）；Purple Mountain Laboratories（紫金山实验室）；Ocean University of China（中国海洋大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Anhui University（安徽大学）；The University of Sydney（悉尼大学）；JD Explore Academy（京东探索研究院）
With the development of convolutional neural networks, hundreds of deep learning based dehazing methods have been proposed. In this paper, we provide a comprehensive survey on supervised, semi-supervised, and unsupervised single image dehazing. We first discuss the physical model, datasets, network modules, loss functions, and evaluation metrics that are commonly used. Then, the main contributions of various dehazing algorithms are categorized and summarized. Further, quantitative and qualitative experiments of various baseline methods are carried out. Finally, the unsolved issues and challeng…
Indira Gandhi Delhi Technical University for Women（英迪拉·甘地德里女子技术大学）
As humans, we can remember certain visuals in great detail, and sometimes even after viewing them once. What is even more interesting is that humans tend to remember and forget the same things, suggesting that there might be some general internal characteristics of an image to encode and discard similar types of information. Research suggests that some pictures tend to be memorized more than others. The ability of an image to be remembered by different viewers is one of its intrinsic properties. In visualization and photography, creating memorable images is a difficult task. Hence, to solve th…
Preference Alignment on Diffusion Model: A Comprehensive Survey for Image Generation and Editing
University of Liverpool（利物浦大学）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Edinburgh（爱丁堡大学）；University of Bristol（布里斯托大学）；University of Exeter（埃克塞特大学）
The last decade, (2012 - 2022), saw an unprecedented advance in machine learning (ML) techniques, particularly deep learning (DL). As a result of the proven capabilities of DL, a large amount of work has been presented and studied in almost every field. Since 2012, when the convolution neural networks have been reintroduced in the context of \textit{ImagNet} competition, DL continued to achieve superior performance in many challenging tasks and problems. Wireless communications, in general, and medium access control (MAC) techniques, in particular, were among the fields that were heavily affec…
University of Verona（维罗纳大学）；University of Florida（佛罗里达大学）
In the last decade, Human Activity Recognition (HAR) has become a vibrant research area, especially due to the spread of electronic devices such as smartphones, smartwatches and video cameras present in our daily lives. In addition, the advance of deep learning and other machine learning algorithms has allowed researchers to use HAR in various domains including sports, health and well-being applications. For example, HAR is considered as one of the most promising assistive technology tools to support elderly's daily life by monitoring their cognitive and physical function through daily activit…
A Comprehensive Survey On Client Selections in Federated Learning
College of Computing and Information Technology, University of Doha for Science and Technology（多哈科技大学计算与信息技术学院）；Electrical Engineering, Qatar University（卡塔尔大学电气工程系）
With the proliferation of smart devices and revolutions in communications, electrical distribution systems are gradually shifting from passive, manually-operated and inflexible ones, to a massively interconnected cyber-physical smart grid to address the energy challenges of the future. However, the integration of several cutting-edge technologies has introduced several security and privacy vulnerabilities due to the large-scale complexity and resource limitations of deployments. Recent research trends have shown that False Data Injection (FDI) attacks are becoming one of the most malicious cyb…
University of Electronic Science and Technology of China（电子科技大学）
Over the past decade, domain adaptation has become a widely studied branch of transfer learning that aims to improve performance on target domains by leveraging knowledge from the source domain. Conventional domain adaptation methods often assume access to both source and target domain data simultaneously, which may not be feasible in real-world scenarios due to privacy and confidentiality concerns. As a result, the research of Source-Free Domain Adaptation (SFDA) has drawn growing attention in recent years, which only utilizes the source-trained model and unlabeled target data to adapt to the…
Biological neural networks continue to inspire breakthroughs in neural network performance. And yet, one key area of neural computation that has been under-appreciated and under-investigated is biologically plausible, energy-efficient spiking neural networks, whose potential is especially attractive for low-power, mobile, or otherwise hardware-constrained settings. We present a literature review of recent developments in the interpretation, optimization, efficiency, and accuracy of spiking neural networks. Key contributions include identification, discussion, and comparison of cutting-edge met…
