arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.04281cs.CLcs.AI
收藏

利用LLM识别相似数据点:一种使用摘要与隐藏状态洞察的人在回路策略

Similar Data Points Identification with LLM: A Human-in-the-loop Strategy Using Summarization and Hidden State Insights

Ohio University(俄亥俄大学);Brown University(布朗大学);Rutgers University(罗格斯大学)

Xianlong Zeng, Yijing Gao, Fanghao Song, Ang Liu

AI总结:

本文提出一种人在回路策略,通过LLM摘要与隐藏状态提取两步法识别表格和图像等非自由文本数据中的相似数据点,并在多个数据集上验证其有效性,支持非技术专家快速识别特定场景下的相似数据点。

摘要:

本研究引入了一种简单而有效的方法,用于在非自由文本领域(如表格和图像数据)中利用大型语言模型(LLMs)识别相似数据点。我们的两步方法包括数据点摘要和隐藏状态提取。首先,通过使用LLM进行摘要来压缩数据,降低复杂性并在句子中突出关键信息。随后,将摘要句子输入另一个LLM以提取隐藏状态,作为紧凑且特征丰富的表示。该方法利用了LLM先进的理解和生成能力,为跨不同数据集的相似性识别提供了一种可扩展且高效的策略。我们在多个数据集上证明了该方法在识别相似数据点方面的有效性。此外,我们的方法使非技术领域专家(如欺诈调查员或营销运营人员)能够快速识别针对特定场景定制的相似数据点,展示了其在实际应用中的实用性。总体而言,我们的结果为在各个领域的数据分析中利用LLM开辟了新途径。

英文摘要:

This study introduces a simple yet effective method for identifying similar data points across non-free text domains, such as tabular and image data, using Large Language Models (LLMs). Our two-step approach involves data point summarization and hidden state extraction. Initially, data is condensed via summarization using an LLM, reducing complexity and highlighting essential information in sentences. Subsequently, the summarization sentences are fed through another LLM to extract hidden states, serving as compact, feature-rich representations. This approach leverages the advanced comprehension and generative capabilities of LLMs, offering a scalable and efficient strategy for similarity identification across diverse datasets. We demonstrate the effectiveness of our method in identifying similar data points on multiple datasets. Additionally, our approach enables non-technical domain experts, such as fraud investigators or marketing operators, to quickly identify similar data points tailored to specific scenarios, demonstrating its utility in practical applications. In general, our results open new avenues for leveraging LLMs in data analysis across various domains

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑