ZeroR@CHiPSAL 2026：用于尼泊尔表情包分类的对比学习两阶段视觉-语言适配
ZeroR@CHiPSAL 2026: Two-Stage Vision-Language Adaptation with Contrastive Learning for Nepali Meme Classification
浏览论文内容
中文总结 AI 辅助
该研究针对尼泊尔表情包多模态仇恨言论与情感检测任务，适配RA-HMD框架，采用两阶段视觉-语言对比学习方法，取得两项任务的优异排名，为低资源南亚语言适配大模型提供了见解。
中文摘要 AI 辅助
本文介绍了我们针对CHiPSAL 2026共享任务的系统，该任务聚焦尼泊尔表情包中的多模态仇恨言论与情感检测，涵盖仇恨言论二分类和情感三分类两个子任务。我们采用具备天城文原生支持的先进视觉-语言模型Qwen3-VL-8B-Instruct，对Robust Adaptation of Hateful Meme Detection（RA-HMD）框架进行适配。我们采用两阶段训练流程：（1）带MLP投影头的LoRA微调用于生成式分类；（2）带监督式InfoNCE损失的骨干网络对比微调。我们通过少数类过采样、图像增强和焦点损失处理类别不平衡问题。推理阶段，我们用验证集调优的权重集成第一阶段的token概率与第二阶段的分类器得分，借助模型的天城文原生理解能力，消除了独立OCR与翻译流程带来的误差传播。我们的系统在仇恨言论检测任务中取得第2名（F1值：0.797），在情感分析任务中取得第4名（F1值：0.518）。我们提供了详细的 ablation 实验、误差分析，以及关于为低资源南亚语言适配大型视觉-语言模型的见解。
英文摘要
This paper presents our system for the CHiPSAL 2026 shared task on multimodal hate speech and sentiment detection in Nepali memes. We address both subtasks: binary hate speech classification and three-class sentiment analysis. Our approach adapts the Robust Adaptation of Hateful Meme Detection (RA-HMD) framework using Qwen3-VL-8B-Instruct, a state-of-the-art vision-language model with native Devanagari support. We employ a two-stage training pipeline: (1) LoRA fine-tuning with an MLP projection head for generative classification, and (2) contrastive backbone fine-tuning with supervised InfoNCE loss. We handle class imbalance through minority oversampling, image augmentation, and focal loss. At inference, we ensemble Stage 1 token probabilities with Stage 2 classifier scores using validation-tuned weights. Our end-to-end approach eliminates error propagation from separate OCR and translation pipelines by leveraging the model's native Devanagari understanding. Our system achieved \textbf{2nd place} on hate speech detection (F1: 0.797) and \textbf{4th place} on sentiment analysis (F1: 0.518). We provide detailed ablations, error analysis, and insights into adapting large vision-language models for low-resource South Asian languages.
发表机构
- Pulchowk Campus, Institute of Engineering, Tribhuvan University（特里布万大学工程学院普尔乔克校区）
机构由 AI 辅助整理，请以论文原文为准。
