超越标签的推理：在低资源、文化细腻情境中衡量LLM情感
Reasoning Beyond Labels: Measuring LLM Sentiment in Low-Resource, Culturally Nuanced Contexts
AI总结：
本研究提出诊断框架，将情感视为文化嵌入构念，评估LLM在低资源、语码混合的WhatsApp消息中的推理，发现顶级模型稳定而开放模型易受模糊性影响，强调文化敏感的AI评估。
摘要：
在低资源、文化细腻的情境中进行情感分析，挑战了假设固定标签和普遍情感表达的传统NLP方法。我们提出了一个诊断框架，将情感视为依赖于上下文、文化嵌入的构念，并评估大型语言模型（LLMs）如何在内罗毕青年健康群体的非正式、语码混合的WhatsApp消息中推理情感。通过结合人工标注数据、情感翻转的反事实样本和基于量规的解释评估，我们探究了LLM的可解释性、鲁棒性以及与人类推理的一致性。通过社会科学测量视角构建我们的评估，我们将LLM输出操作化并审视为测量抽象情感概念的工具。我们的研究结果揭示了模型推理质量的显著差异，顶级LLM表现出解释稳定性，而开放模型在模糊性或情感转变下往往表现不佳。这项工作强调了在复杂现实通信中，需要文化敏感、推理感知的AI评估。
英文摘要：
Sentiment analysis in low-resource, culturally nuanced contexts challenges conventional NLP approaches that assume fixed labels and universal affective expressions. We present a diagnostic framework that treats sentiment as a context-dependent, culturally embedded construct, and evaluate how large language models (LLMs) reason about sentiment in informal, code-mixed WhatsApp messages from Nairobi youth health groups. Using a combination of human-annotated data, sentiment-flipped counterfactuals, and rubric-based explanation evaluation, we probe LLM interpretability, robustness, and alignment with human reasoning. Framing our evaluation through a social-science measurement lens, we operationalize and interrogate LLMs outputs as an instrument for measuring the abstract concept of sentiment. Our findings reveal significant variation in model reasoning quality, with top-tier LLMs demonstrating interpretive stability, while open models often falter under ambiguity or sentiment shifts. This work highlights the need for culturally sensitive, reasoning-aware AI evaluation in complex, real-world communication.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
