GATech at AbjadMed: 双向编码器与因果解码器：来自82类阿拉伯语医学分类的见解
GATech at AbjadMed: Bidirectional Encoders vs. Causal Decoders: Insights from 82-Class Arabic Medical Classification
AI总结：
本文研究了阿拉伯语医学文本在82类分类中的表现，对比了双向编码器与因果解码器，发现前者在捕捉细粒度语义边界方面更优。
摘要：
本文介绍了针对82个不同类别的阿拉伯语医学文本分类系统的架构描述。我们的主要架构采用了一个经过微调的AraBERTv2编码器，并结合了注意力和均值表示的混合池化策略，以及多样本dropout进行鲁棒正则化。我们系统地将此方法与多种多语言和阿拉伯语特定的编码器，以及几种大规模因果解码器进行比较，包括通过Llama 3.3 70B进行的零样本重排序和从Qwen 3B隐藏状态中提取的特征。我们的发现表明，专门设计的双向编码器在捕捉细粒度医学文本分类所需的精确语义边界方面显著优于因果解码器。我们展示了因果解码器，优化于下一个标记预测，会产生具有序列偏见的嵌入，其在分类任务中不如双向注意力捕捉的全局上下文有效。尽管在训练数据中发现了显著的类别不平衡和标签噪声，我们的结果突显了微调编码器在专门阿拉伯语NLP任务中的优越语义压缩能力。测试集上的最终性能指标，包括准确率和宏F1值，均被报告并讨论。
英文摘要：
This paper presents system description for Arabic medical text classification across 82 distinct categories. Our primary architecture utilizes a fine-tuned AraBERTv2 encoder enhanced with a hybrid pooling strategies, combining attention and mean representations, and multi-sample dropout for robust regularization. We systematically benchmark this approach against a suite of multilingual and Arabic-specific encoders, as well as several large-scale causal decoders, including zero-shot re-ranking via Llama 3.3 70B and feature extraction from Qwen 3B hidden states. Our findings demonstrate that specialized bidirectional encoders significantly outperform causal decoders in capturing the precise semantic boundaries required for fine-grained medical text classification. We show that causal decoders, optimized for next-token prediction, produce sequence-biased embeddings that are less effective for categorization compared to the global context captured by bidirectional attention. Despite significant class imbalance and label noise identified within the training data, our results highlight the superior semantic compression of fine-tuned encoders for specialized Arabic NLP tasks. Final performance metrics on the test set, including Accuracy and Macro-F1, are reported and discussed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
