首页/论文/2101.06166

arXiv 2101.061662021-01-15cs.LG

A General Framework for Hypercomplex-valued Extreme Learning Machines

University of Campinas（坎皮纳斯大学）

Guilherme Vieira, Marcos Eduardo Valle

发表信息：: Journal of Computational Mathematics and Data Science, Volume 3, June 2022. Article 100032
DOI：: 10.1016/j.jcmds.2022.100032

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

This paper aims to establish a framework for extreme learning machines (ELMs) on general hypercomplex algebras. Hypercomplex neural networks are machine learning models that feature higher-dimension numbers as parameters, inputs, and outputs. Firstly, we review broad hypercomplex algebras and show a framework to operate in these algebras through real-valued linear algebra operations in a robust manner. We proceed to explore a handful of well-known four-dimensional examples. Then, we propose the hypercomplex-valued ELMs and derive their learning using a hypercomplex-valued least-squares problem. Finally, we compare real and hypercomplex-valued ELM models' performance in an experiment on time-series prediction and another on color image auto-encoding. The computational experiments highlight the excellent performance of hypercomplex-valued ELMs to treat high-dimensional data, including models based on unusual hypercomplex algebras.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

扩展一大类超复数值神经网络的万能逼近定理

Extending the Universal Approximation Theorem for a Broad Class of Hypercomplex-Valued Neural Networks

Universidade Estadual de Campinas（坎皮纳斯州立大学）

本文通过引入非退化超复代数概念，将万能逼近定理扩展至一大类超复数值神经网络，涵盖复数、四元数和tessarines等代数，为该类网络的存在性提供了理论支撑。

2022-09-06cs.LG

向量值与超复数值神经网络的通用逼近定理

Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks

Universidade Estadual de Campinas (UNICAMP)（坎皮纳斯州立大学）；Instituto de Pesquisa Eldorado（埃尔多拉多研究所）

本文引入非退化代数概念，将通用逼近定理从实值及部分超复数值神经网络推广到广泛的向量值神经网络。

2024-08-09cs.LGcs.NE

Hypercomplex-Valued Recurrent Correlation Neural Networks

Institute of Mathematics, Statistics, and Scientific Computing（数学、统计与科学计算研究所）；University of Campinas（坎皮纳斯大学）

Recurrent correlation neural networks (RCNNs), introduced by Chiueh and Goodman as an improved version of the bipolar correlation-based Hopfield neural network, can be used to implement high-capacity associative memories. In this paper, we extend the bipolar RCNNs for processing hypercomplex-valued data. Precisely, we present the mathematical background for a broad class of hypercomplex-valued RCNNs. Then, we provide the necessary conditions which ensure that a hypercomplex-valued RCNN always settles at an equilibrium using either synchronous or asynchronous update modes. Examples with bipolar…

2020-01-31cs.LGcs.NEstat.ML

A Broad Class of Discrete-Time Hypercomplex-Valued Hopfield Neural Networks

Federal Institute of Education, Science and Technology of Espírito Santo（圣埃斯皮里图联邦教育、科学与技术学院）；University of Campinas（坎皮纳斯大学）

In this paper, we address the stability of a broad class of discrete-time hypercomplex-valued Hopfield-type neural networks. To ensure the neural networks belonging to this class always settle down at a stationary state, we introduce novel hypercomplex number systems referred to as real-part associative hypercomplex number systems. Real-part associative hypercomplex number systems generalize the well-known Cayley-Dickson algebras and real Clifford algebras and include the systems of real numbers, complex numbers, dual numbers, hyperbolic numbers, quaternions, tessarines, and octonions as parti…

2019-10-31cs.LGmath.DSstat.ML

理解向量值神经网络及其与实值和超复值神经网络的关系

Understanding Vector-Valued Neural Networks and Their Relationship with Real and Hypercomplex-Valued Neural Networks

Universidade Estadual de Campinas (UNICAMP)（坎皮纳斯州立大学）

本文提出向量值神经网络框架 V-nets，通过自然建模特征通道间相关性减少参数并提升训练稳健性，并阐明其与传统实值网络及超复值网络的关系和实现方法。

2024-08-01cs.LGcs.NE

超复数值神经网络的全张量方法

Fully tensorial approach to hypercomplex-valued neural networks

Faculty of Computer Science and Mathematics, Cracow University of Technology（克拉科夫技术大学计算机科学与数学系）

本文提出基于张量的理论框架，使神经网络能在任意有限维代数上操作，通过张量运算统一描述密集和卷积层，并为超复数值感知机提供理论基础。

2026-01-26cs.AIcs.LGcs.NE

更多相关论文收起更多论文

使用超复数值卷积神经网络进行急性淋巴细胞白血病检测

Acute Lymphoblastic Leukemia Detection Using Hypercomplex-Valued Convolutional Neural Networks

University of Campinas（坎皮纳斯大学）

本文提出基于超复数代数的卷积神经网络（HvCNNs）用于血液涂片淋巴细胞分类以辅助诊断急性淋巴细胞白血病，实验表明其在参数更少的情况下准确率优于实数值模型，基于Clifford代数处理HSV图像的HvCNN在ALL-IDB2数据集上达96.6%准确率，接近最先进水平。

2022-05-26cs.CVcs.LGcs.NE

超复数广义线性处理：四元数机器学习的基础

Hypercomplex Widely Linear Processing: Fundamentals for Quaternion Machine Learning

本文介绍了四元数机器学习的基础，包括四元数代数、广义线性模型、增强统计量和均方估计，旨在为四元数在机器学习中的应用提供理论支持和实践指导。

2026-03-12cs.LGstat.ML

揭秘超复数：超复数深度学习中的归纳偏置

Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning

Sapienza University of Rome（罗马大学）；Imperial College London（帝国理工学院）

本文提出统一的归纳偏置理论框架，解释超复数深度学习成功的原因，并论证其可作为多维与多模态信号处理中实值深度学习的可行替代方案。

2024-05-11cs.LGeess.SP

PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions

Sapienza University of Rome（罗马大学）；Amazon Web Services AI（亚马逊云服务人工智能部门）

Hypercomplex neural networks have proven to reduce the overall number of parameters while ensuring valuable performance by leveraging the properties of Clifford algebras. Recently, hypercomplex linear layers have been further improved by involving efficient parameterized Kronecker products. In this paper, we define the parameterization of hypercomplex convolutional layers and introduce the family of parameterized hypercomplex neural networks (PHNNs) that are lightweight and efficient large-scale models. Our method grasps the convolution rules and the filter organization directly from data with…

2022-09-19cs.AIcs.CVcs.LG

Quaternion-Valued Convolutional Neural Network Applied for Acute Lymphoblastic Leukemia Diagnosis

Escuela Superior Politécnica del Litoral（滨海理工学院）；Universidade Estadual de Campinas（坎皮纳斯州立大学）

The field of neural networks has seen significant advances in recent years with the development of deep and convolutional neural networks. Although many of the current works address real-valued models, recent studies reveal that neural networks with hypercomplex-valued parameters can better capture, generalize, and represent the complexity of multidimensional data. This paper explores the quaternion-valued convolutional neural network application for a pattern recognition task from medicine, namely, the diagnosis of acute lymphoblastic leukemia. Precisely, we compare the performance of real-va…

2021-12-13cs.CVcs.LGcs.NE

Knowledge Graph Embeddings in Geometric Algebras

University of Bonn（波恩大学）；Fraunhofer IAIS（弗劳恩霍夫智能分析与信息系统研究所）

Knowledge graph (KG) embedding aims at embedding entities and relations in a KG into a lowdimensional latent representation space. Existing KG embedding approaches model entities andrelations in a KG by utilizing real-valued , complex-valued, or hypercomplex-valued (Quaternionor Octonion) representations, all of which are subsumed into a geometric algebra. In this work,we introduce a novel geometric algebra-based KG embedding framework, GeomE, which uti-lizes multivector representations and the geometric product to model entities and relations. Ourframework subsumes several state-of-the-art KG…

2021-03-22cs.AIcs.LGstat.ML

Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling

The University of Queensland（昆士兰大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Hefei University of Technology（合肥工业大学）

As a well-established approach, factorization machine (FM) is capable of automatically learning high-order interactions among features to make predictions without the need for manual feature engineering. With the prominent development of deep neural networks (DNNs), there is a recent and ongoing trend of enhancing the expressiveness of FM-based models with DNNs. However, though better results are obtained with DNN-based FM variants, such performance gain is paid off by an enormous amount (usually millions) of excessive model parameters on top of the plain FM. Consequently, the heavy parameteri…

2021-04-11cs.IRcs.LG

Fairy2i: 从真实LLM中训练复杂LLM，所有参数在{±1, ±i}中

Fairy2i: Training Complex LLMs from Real LLMs with All Parameters in $\{\pm 1, \pm i\}$

Peking University（北京大学）

Fairy2i通过将预训练实值层转换为复形式，实现极低比特量化，显著提升LLM推理效率。

2026-01-29cs.AIcs.LG

二元二次量化：面向实值矩阵压缩的超越一阶量化方法

Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression

Institute of Science Tokyo（东京科学研究所）；Waseda University（早稻田大学）

该研究提出二元二次量化（BQQ）方法，利用二元二次表达式提升矩阵压缩效果，在矩阵压缩基准和Vision Transformer的2比特训练后量化实验中，性能均优于传统方法，验证了该表达式在高效矩阵近似与神经网络压缩中的有效性。

2025-10-21cs.AIcs.CVcs.LG