arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.05481cs.LG
收藏

fLSA:使用基础模型学习文档集合中的语义结构

fLSA: Learning Semantic Structures in Document Collections Using Foundation Models

Microsoft Research Redmond(微软研究院雷德蒙德)

Weijia Xu, Nebojsa Jojic, Nicolas Le Roux

AI总结:

针对能否用大型语言模型从示例文档归纳高级结构的问题,提出基于基础模型的潜在语义分析方法fLSA,其通过迭代聚类和标记文档片段建模潜在结构,实验证明在重构文本和分层采样方面优于现有方法。

摘要:

人类可通过从相似问题的示例解决方案中归纳高级策略,再调整这些策略解决未见过的问题来学习新任务。我们能否用大型语言模型从示例文档或解决方案中归纳此类高级结构?我们提出fLSA,一种基于基础模型的潜在语义分析方法,它基于文档级上下文迭代聚类和标记文档片段。这些标记可用于建模给定文档的潜在结构,以及对新文本进行分层采样。我们在故事写作、数学和多步推理数据集上的实验表明,与现有标记方法相比,fLSA标记在重构原始文本时信息更丰富;此外,用于分层采样时,fLSA标记比直接采样和使用现有标记方法的分层采样更常引导输出空间向正确方向扩展,从而得到正确解决方案。代码:https://github.com/microsoft/fLSA

英文摘要:

Humans can learn to solve new tasks by inducing high-level strategies from example solutions to similar problems and then adapting these strategies to solve unseen problems. Can we use large language models to induce such high-level structure from example documents or solutions? We introduce fLSA, a foundation-model-based Latent Semantic Analysis method that iteratively clusters and tags document segments based on document-level contexts. These tags can be used to model the latent structure of given documents and for hierarchical sampling of new texts. Our experiments on story writing, math, and multi-step reasoning datasets demonstrate that fLSA tags are more informative in reconstructing the original texts than existing tagging methods. Moreover, when used for hierarchical sampling, fLSA tags help expand the output space in the right directions that lead to correct solutions more often than direct sampling and hierarchical sampling with existing tagging methods. Code: https://github.com/microsoft/fLSA

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑