LinGO：一种结合大语言模型的语言图优化框架，用于解释在线不文明言论的意图
LinGO: A Linguistic Graph Optimization Framework with LLMs for Interpreting Intents of Online Uncivil Discourse
AI总结：
LinGO通过多步骤语言组件优化，提升大语言模型对在线不文明言论意图的识别与解释能力。
摘要：
检测不文明语言对于维护安全、包容和民主的在线空间至关重要。然而，现有的分类器常常错误地解读包含不文明线索但表达文明意图的帖子，导致在线有害不文明现象的估计被高估。我们介绍了LinGO，一种针对大语言模型（LLMs）的语言图优化框架，利用语言结构和优化技术来分类使用各种直接和间接表达的多类不文明意图。LinGO将语言分解为多步骤语言组件，识别导致最多错误的针对性步骤，并迭代优化提示和/或示例组件以针对这些步骤。我们使用2022年巴西总统选举期间收集的数据集进行评估，该数据集涵盖了四种形式的政治不文明：不礼貌（IMP）、仇恨言论和刻板印象（HSST）、身体伤害和暴力政治修辞（PHAVPR）以及对民主机构和价值观的威胁（THREAT）。每个实例都被标注为六种类型的文明/不文明意图。我们使用三种成本效益高的LLM：GPT-5-mini、Gemini 2.5 Flash-Lite和Claude 3 Haiku，以及四种优化技术：TextGrad、AdalFlow、DSPy和检索增强生成（RAG）进行基准测试。结果表明，在所有模型中，LinGO在准确率和加权F1方面均优于零样本、链式思维、直接优化和微调基线。RAG是最强的优化技术，当与Gemini模型结合时，实现了最佳整体性能。这些发现表明，将多步骤语言组件纳入LLM指令并优化针对性组件可以帮助模型解释复杂的语义含义，这可以扩展到其他复杂的语义解释任务。
英文摘要：
Detecting uncivil language is crucial for maintaining safe, inclusive, and democratic online spaces. Yet existing classifiers often misinterpret posts containing uncivil cues but expressing civil intents, leading to inflated estimates of harmful incivility online. We introduce LinGO, a linguistic graph optimization framework for large language models (LLMs) that leverages linguistic structures and optimization techniques to classify multi-class intents of incivility that use various direct and indirect expressions. LinGO decomposes language into multi-step linguistic components, identifies targeted steps that cause the most errors, and iteratively optimizes prompt and/or example components for targeted steps. We evaluate it using a dataset collected during the 2022 Brazilian presidential election, encompassing four forms of political incivility: Impoliteness (IMP), Hate Speech and Stereotyping (HSST), Physical Harm and Violent Political Rhetoric (PHAVPR), and Threats to Democratic Institutions and Values (THREAT). Each instance is annotated with six types of civil/uncivil intent. We benchmark LinGO using three cost-efficient LLMs: GPT-5-mini, Gemini 2.5 Flash-Lite, and Claude 3 Haiku, and four optimization techniques: TextGrad, AdalFlow, DSPy, and Retrieval-Augmented Generation (RAG). The results show that, across all models, LinGO consistently improves accuracy and weighted F1 compared with zero-shot, chain-of-thought, direct optimization, and fine-tuning baselines. RAG is the strongest optimization technique and, when paired with Gemini model, achieves the best overall performance. These findings demonstrate that incorporating multi-step linguistic components into LLM instructions and optimize targeted components can help the models explain complex semantic meanings, which can be extended to other complex semantic explanation tasks in the future.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
