CRAFT：从开放环境中提取并调优文化指令
CRAFT: Extracting and Tuning Cultural Instructions from the Wild
AI总结：
针对大语言模型文化推理能力不足的问题，本文提出CRAFT流水线，从非结构化语料提取文化指令调优数据，在三个地区实验中实现最高6%的性能提升，为文化指令调优集构建提供了新方向。
摘要：
大语言模型（LLM）已快速发展为各类自然语言处理（NLP）应用的基础。尽管其应用场景广泛，但对文化相关概念的理解与推理能力仍存在局限。同时，提升这类模型的文化推理能力存在迫切需求，尤其是针对代表性不足的地区。本文提出一种新型流水线，可从海量非结构化语料中提取高质量的文化相关指令调优数据集。我们采用自指令生成流水线来识别文化概念并触发指令生成。通过与通用指令调优数据集融合，我们的模型在识别和理解地域文化细微差异方面的能力得到提升，进而增强了推理能力。我们在新加坡、菲律宾和美国三个地区开展实验，性能提升最高达6%。本研究为直接从非结构化数据中提取文化指令调优集开辟了新路径，为该领域的未来创新树立了先例。
英文摘要：
Large language models (LLMs) have rapidly evolved as the foundation of various natural language processing (NLP) applications. Despite their wide use cases, their understanding of culturally-related concepts and reasoning remains limited. Meantime, there is a significant need to enhance these models' cultural reasoning capabilities, especially concerning underrepresented regions. This paper introduces a novel pipeline for extracting high-quality, culturally-related instruction tuning datasets from vast unstructured corpora. We utilize a self-instruction generation pipeline to identify cultural concepts and trigger instruction. By integrating with a general-purpose instruction tuning dataset, our model demonstrates enhanced capabilities in recognizing and understanding regional cultural nuances, thereby enhancing its reasoning capabilities. We conduct experiments across three regions: Singapore, the Philippines, and the United States, achieving performance improvement of up to 6%. Our research opens new avenues for extracting cultural instruction tuning sets directly from unstructured data, setting a precedent for future innovations in the field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
