概念归纳:使用LLooM通过高层概念分析非结构化文本
Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM
AI总结:
本文提出概念归纳算法LLooM,利用大型语言模型从非结构化文本中生成高层级、可解释的概念,优于主题模型,并支持理论驱动的分析。
摘要:
数据分析师长期以来一直寻求将非结构化文本数据转化为有意义的概念。尽管主题建模和聚类很常见,但它们侧重于较低层级的关键词,并需要大量的解释性工作。我们引入了概念归纳(concept induction),这是一种计算过程,能够从非结构化文本中生成由明确包含标准定义的高层概念。对于有毒在线评论的数据集,当最先进的BERTopic模型输出“女性、权力、女性”时,概念归纳生成了诸如“对传统性别角色的批评”和“对女性关切的忽视”等高层次概念。我们提出了LLooM,一种概念归纳算法,利用大型语言模型迭代地综合采样文本,并提出越来越普遍的人类可解释概念。然后,我们将LLooM实例化到一个混合主动的文本分析工具中,使分析师能够将注意力从解释主题转向进行理论驱动的分析。通过技术评估和从文献综述到内容审核的四个分析场景,我们发现LLooM的概念在质量和数据覆盖率方面优于先前的主题模型技术。在专家案例研究中,LLooM帮助研究人员即使从熟悉的数据集中也发现了新的见解,例如在政治社交媒体数据集中提出了一个先前未被注意到的针对外部政党立场的攻击概念。
英文摘要:
Data analysts have long sought to turn unstructured text data into meaningful concepts. Though common, topic modeling and clustering focus on lower-level keywords and require significant interpretative work. We introduce concept induction, a computational process that instead produces high-level concepts, defined by explicit inclusion criteria, from unstructured text. For a dataset of toxic online comments, where a state-of-the-art BERTopic model outputs "women, power, female," concept induction produces high-level concepts such as "Criticism of traditional gender roles" and "Dismissal of women's concerns." We present LLooM, a concept induction algorithm that leverages large language models to iteratively synthesize sampled text and propose human-interpretable concepts of increasing generality. We then instantiate LLooM in a mixed-initiative text analysis tool, enabling analysts to shift their attention from interpreting topics to engaging in theory-driven analysis. Through technical evaluations and four analysis scenarios ranging from literature review to content moderation, we find that LLooM's concepts improve upon the prior art of topic models in terms of quality and data coverage. In expert case studies, LLooM helped researchers to uncover new insights even from familiar datasets, for example by suggesting a previously unnoticed concept of attacks on out-party stances in a political social media dataset.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。