Expedition:一个概念层次的无监督学习系统
Expedition: A System for the Unsupervised Learning of a Hierarchy of Concepts
摘要:
我们提出了一个系统自下而上的累积学习的无数概念对应的有意义的字符串,其部分相关的和预测的边缘。学习是自我监督的,因为发现的概念被用作预测器和预测目标。我们设计了一个目标分割与学习的概念,来自比较基线预测系统,促进制作和使用更大的概念,这反过来又允许预测更大的文本跨度,我们描述了一个简单的技术,以促进探索,即尝试新生成的概念在分割过程中。我们激励和解释的概念分层,以帮助分离的(条件)分布之间的概念。概念的分层大致对应于部分-整体概念层次结构。由于具有基本的分割和学习算法,该系统很有前途,因为它获得了许多概念(在我们的小规模实验中有数万个),它学会了很好地分割文本:当输入去除了空格的英文文本时,从字符级开始,所学习的大部分内容尊重单词或短语边界,并且随着时间的推移,分割内的“坏”分割的平均数量,即在词内分裂,随着发现更大的概念而减小,并且系统学习在分割期间何时使用它们。我们报告有前途的实验时,输入文本转换为二进制和系统开始只有两个概念,“0”和“1”。该系统是透明的,在这个意义上,它很容易告诉学习的概念对应于什么,哪些是活跃的分割,或系统如何“看到”它的输入。我们希望这个框架是可扩展的,我们讨论了目前的限制和一些方向,以提高学习和推理能力。
英文摘要:
We present a system for bottom-up cumulative learning of myriad concepts corresponding to meaningful character strings, and their part-related and prediction edges. The learning is self-supervised in that the concepts discovered are used as predictors as well as targets of prediction. We devise an objective for segmenting with the learned concepts, derived from comparing to a baseline prediction system, that promotes making and using larger concepts, which in turn allows for predicting larger spans of text, and we describe a simple technique to promote exploration, i.e. trying out newly generated concepts in the segmentation process. We motivate and explain a layering of the concepts, to help separate the (conditional) distributions learnt among concepts. The layering of the concepts roughly corresponds to a part-whole concept hierarchy. With rudimentary segmentation and learning algorithms, the system is promising in that it acquires many concepts (tens of thousands in our small-scale experiments), and it learns to segment text well: when fed with English text with spaces removed, starting at the character level, much of what is learned respects word or phrase boundaries, and over time the average number of "bad" splits within segmentations, i.e. splits inside words, decreases as larger concepts are discovered and the system learns when to use them during segmentation. We report on promising experiments when the input text is converted to binary and the system begins with only two concepts, "0" and "1". The system is transparent, in the sense that it is easy to tell what the concepts learned correspond to, and which ones are active in a segmentation, or how the system "sees" its input. We expect this framework to be extensible and we discuss the current limitations and a number of directions for enhancing the learning and inference capabilities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。