Learning is a process wherein a learning agent enhances its performance through exposure of experience or data. Throughout this journey, the agent may encounter diverse learning environments. For example, data may be presented to the leaner all at once, in multiple batches, or sequentially. Furthermore, the distribution of each data sample could be either identical and independent (iid) or non-iid. Additionally, there may exist computational and space constraints for the deployment of the learning algorithms. The complexity of a learning task can vary significantly, depending on the learning setup and the constraints imposed upon it. However, it is worth noting that the current literature lacks formal definitions for many of the in-distribution and out-of-distribution learning paradigms. Establishing proper and universally agreed-upon definitions for these learning setups is essential for thoroughly exploring the evolution of ideas across different learning scenarios and deriving generalized mathematical bounds for these learners. In this paper, we aim to address this issue by proposing a chronological approach to defining different learning tasks using the provably approximately correct (PAC) learning framework. We will start with in-distribution learning and progress to recently proposed lifelong or continual learning. We employ consistent terminology and notation to demonstrate how each of these learning frameworks represents a specific instance of a broader, more generalized concept of learnability. Our hope is that this work will inspire a universally agreed-upon approach to quantifying different types of learning, fostering greater understanding and progress in the field.
Federated Distillation on Edge Devices: Efficient Client-Side Filtering for Non-IID Data
1 Embedded Systems Division, Silicon Austria Labs, Graz, Austria；2 Department of Computer Science, University of Innsbruck, Austria；5 Institute of Information Technology, University of Klagenfurt, Austria；3 TU-Graz SAL DES Lab, Silicon Austria Labs, Graz, Austria；4 Institute of Visual Computing, Graz University of Technology, Austria
Homophily Outlier Detection in Non-IID Categorical Data
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Center of Artificial Intelligence, University of Technology Sydney（悉尼科技大学人工智能中心）；Data Science Lab, University of Technology Sydney（悉尼科技大学数据科学实验室）
LoAdaBoost: loss-based AdaBoost federated machine learning with reduced computational complexity on IID and non-IID intensive care data
Academy of Arts and Design, Tsinghua University（清华大学美术学院）；The Future Laboratory, Tsinghua University（清华大学未来实验室）；University of Huddersfield（哈德斯菲尔德大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Northeastern University（东北大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Fed-TDA: Federated Tabular Data Augmentation on Non-IID Data
School of Computer Science, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院）；Department of New Networks, Peng Cheng Laboratory（鹏城实验室新型网络部）
Robust and Communication-Efficient Federated Learning from Non-IID Data
Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Korea University（高丽大学）
Clust-PSI-PFL: A Population Stability Index Approach for Clustered Non-IID Personalized Federated Learning
Department of Computer, Control；Management Engineering Sapienza University of Rome Rome, Italy；Telef\'onica Research Barcelona, Spain；Keysight AI Labs Barcelona, Spain
