Quantum machine learning models generally lack principled design guidelines, often requiring full resource-intensive training across numerous choices of encodings, quantum circuit designs and initialization strategies to find effective configuration. To address this challenge, we develope the Quantum Bias-Expressivity Toolbox ($\texttt{QBET}$), a framework for evaluating quantum, classical, and hybrid transformer architectures. In this toolbox, we introduce lean metrics for Simplicity Bias ($\texttt{SB}$) and Expressivity ($\texttt{EXP}$), for comparing across various models, and extend the analysis of $\texttt{SB}$ to generative and multiclass-classification tasks. We show that $\texttt{QBET}$ enables efficient pre-screening of promising model variants obviating the need to execute complete training pipelines. In evaluations on transformer-based classification and generative tasks we employ a total of $18$ qubits for embeddings ($6$ qubits each for query, key, and value). We identify scenarios in which quantum self-attention variants surpass their classical counterparts by ranking the respective models according to the $\texttt{SB}$ metric and comparing their relative performance.
Hybrid Quantum Vision Transformers for Event Classification in High Energy Physics
University of Florida(佛罗里达大学);Polytechnic University of Catalonia(加泰罗尼亚理工大学);Indian Institute of Technology Bhilai(贝拉理工学院);University of Kansas(堪萨斯大学);University of Alabama(阿拉巴马大学);Software Engineering Institute, Carnegie Mellon University(卡内基梅隆大学软件工程研究所);Technische University of München(慕尼黑工业大学)
Hybrid Convolution and Vision Transformer NAS Search Space for TinyML Image Classification
Chair of Electronic Design Automation, Technical University of Munich(电子设计自动化研究所,慕尼黑技术大学);Department of Computer Science, University of Tübingen(计算机科学系,图宾根大学);Institute of Computer Engineering, TU Wien(计算机工程研究所,维也纳技术大学)
Tree Meets Transformer: A Hybrid Architecture for Scalable Power Allocation in Cell-Free Networks
Smart Networks and Services Joint Undertaking (SNS JU)(智能网络与服务联合体);Italian Ministry of Education and Research (MUR)(意大利教育与研究部);HORIZON-JU-SNS-2022 EU project TIMES(HORIZON-JU-SNS-2022 EU项目TIMES);European Union under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU(欧盟在意大利国家复苏与韧性计划(NRRP)下的NextGenerationEU)
HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language Model
Microsoft Research(微软研究院);UM-SJTU Joint Institute(上海交通大学密歇根联合学院);Princeton University(普林斯顿大学);Carnegie Mellon University(卡内基梅隆大学);Renmin University of China(中国人民大学)
Hybrid Attention-Based Transformer Block Model for Distant Supervision Relation Extraction
College of Information Science and Technology, Donghua University(东华大学信息科学与技术学院);University of Surrey(萨里大学);Southern University of Science and Technology(南方科技大学)
随着各种数字文本信息呈指数级爆炸式增长,如何从海量的非结构化文本信息中有效地获取特定知识成为一个挑战。关系抽取是自然语言处理的基本任务之一,其目的是从给定的文本中抽取实体对之间的语义关系。为了避免人工标注数据集,远距离监督关系抽取(DSRE)被广泛应用,旨在利用知识库自动标注数据集。不幸的是,由于潜在的强假设,这种方法严重遭受错误的标签。为了解决这个问题,我们提出了一个新的框架,使用混合注意力为基础的Transformer块与多实例学习来执行DSRE任务。具体地说,首先使用Transformer块作为句子编码器来捕获句子的句法信息,其主要利用多头自注意从词级提取特征。然后,采用一种更为简洁的句子级注意力机制来构造袋子表示,旨在整合句子的有效信息来有效地表示袋子。在公开数据集New York Times(NYT)上的实验结果表明,该方法在评估数据集上的性能优于现有算法,验证了该模型在DSRE任务中的有效性.
Spatio-Temporal Hybrid Fusion of CAE and SWIn Transformers for Lung Cancer Malignancy Prediction
Concordia University(康考迪亚大学);Concordia Institute for Information Systems Engineering(康考迪亚信息系统工程研究所);Sunnybrook Health Sciences Centre(桑尼布鲁克健康科学中心);University of Toronto(多伦多大学)
Hybrid Vision Transformer and Quantum Convolutional Neural Network for Image Classification
Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);School of Mathematical Sciences, University of Chinese Academy of Sciences(中国科学院大学数学科学学院);State Key Laboratory of Mathematical Science, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院数学科学国家重点实验室)