A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change is interpreted that this observation brings in extra information and, in learning theory, this corresponds to misclassification, or misprediction. In this paper, we lay the foundations of a new theory that allows one to keep control on the probability of change of compression (which maps into the statistical "risk" in learning applications). Under suitable conditions, the cardinality of the compressed set is shown to be a consistent estimator of the probability of change of compression (without any upper limit on the size of the compressed set); moreover, unprecedentedly tight finite-sample bounds to evaluate the probability of change of compression are obtained under a generally applicable condition of preference. All results are usable in a fully agnostic setup, i.e., without requiring any a priori knowledge on the probability distribution of the observations. Not only these results offer a valid support to develop trust in observation-driven methodologies, they also play a fundamental role in learning techniques as a tool for hyper-parameter tuning.
CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimization
King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI)(SDAIA-KAUST数据科学与人工智能卓越中心);Brno University of Technology(布尔诺理工大学);Kempelen Institute of Intelligent Technologies (KInIT)(肯佩伦智能技术研究所)
FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Moscow Institute of Physics and Technology(莫斯科物理技术学院)
在最近的论文FLECS(Agafonov et al,FLECS:A联邦学习Second-Order Framework via Compression and Sketching)中,针对联邦学习问题提出了二阶框架FLECS。该方法利用Hessian草图的压缩,使通信成本低。然而,FLECS的主要瓶颈是没有压缩的梯度通信。在本文中,我们提出了修改的FLECS压缩梯度差异,我们称之为FLECS-CGD(FLECS压缩梯度差异),使其适用于随机优化。在强凸和非凸的情况下提供收敛保证。实验结果表明,该方法具有较好的实用价值.
Bilateral Distribution Compression: Reducing Both Data Size and Dimensionality
School of Mathematics, University of Bristol, Bristol, United Kingdom(布里斯托大学数学系);Mathematical Institute, University of Oxford, Oxford, United Kingdom(牛津大学数学研究所)
Efficient classification using parallel and scalable compressed model and Its application on intrusion detection
College of Computer Science and Technology, Zhejiang University of Technology(浙江工业大学计算机科学与技术学院);School of Software Microelectronics, Peking University(北京大学软件与微电子学院)
Toward Storage-Aware Learning with Compressed Data An Empirical Exploratory Study on JPEG
School of Integrated Technology, Yonsei University, Seoul, Korea(燕子大学整合技术学院);BK21 Graduate Program in Intelligent Semiconductor Technology, Yonsei University, Seoul, Korea(燕子大学智能半导体技术研究生项目);Department of Mobile Systems Engineering, Dankook University, Yongin, Korea(Dankook大学移动系统工程系)
Optimal Rate Adaption in Federated Learning with Compressed Communications
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);School of Computing, Faculty of Science and Engineering, Macquarie University(麦考瑞大学科学与工程学院计算机学院);Peng Cheng Laboratory(鹏城实验室);School of Computing Science, Simon Fraser University(西蒙弗雷泽大学计算科学学院)
OCR for TIFF Compressed Document Images Directly in Compressed Domain Using Text segmentation and Hidden Markov Model
Indian Institute of Information Technology Allahabad(印度理工学院阿拉哈巴德分校)
在当今的技术时代,文档图像在我们的日常生活中发挥着重要和不可或缺的作用,特别是随着COVID-19的激增,数字扫描文档已成为沟通的主要来源,从而避免了任何形式的身体接触感染。扫描文档图像的存储和传输是一项非常占用内存的任务,因此压缩技术被用于在存档和传输之前减小图像大小。为了提取信息或对压缩图像进行操作,我们有两种方法:第一种方法是对图像进行压缩和操作,然后再次压缩图像,以提高存储和传输的效率。另一种方法是使用底层压缩算法的特性直接处理压缩形式的图像,而不涉及解压缩和重新压缩。本文提出了一种新的思路,即在压缩域中直接为CCITT(The International Telegraph and Telephone Consultative Committee,国际电报电话咨询委员会)压缩机打印的TIFF文档图像开发OCR。在将文本区域分割成行和词之后,使用CCITT的三种编码模式(水平、垂直和通过模式)应用HMM进行识别。实验结果表明,光学字符识别的通过模式给出了一个有前途的结果。
Robustness in Compressed Neural Networks for Object Detection
Multimedia Systems Department(多媒体系统系);Gdańsk University of Technology(格但斯克工业大学);Electronics, Telecommunications and Informatics Faculty(电子、电信与信息学院);Polish National Centre for Research and Development (NCBR)(波兰国家研究与发展中心)
Impossibility Results for Grammar-Compressed Linear Algebra
IBM Almaden Research Center(IBM阿尔马登研究中心);Toyota Technological Institute at Chicago(芝加哥丰田理工学院);Saarland University(萨尔兰大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所)