In many real-world learning tasks, it is expensive to acquire a sufficient number of labeled examples for training. This paper investigates methods for reducing annotation cost by `sample selection'. In this approach, during training the learning program examines many unlabeled examples and selects for labeling only those that are most informative at each stage. This avoids redundantly labeling examples that contribute little new information. Our work follows on previous research on Query By Committee, extending the committee-based paradigm to the context of probabilistic classification. We describe a family of empirical methods for committee-based sample selection in probabilistic classification models, which evaluate the informativeness of an example by measuring the degree of disagreement between several model variants. These variants (the committee) are drawn randomly from a probability distribution conditioned by the training set labeled so far. The method was applied to the real-world natural language processing task of stochastic part-of-speech tagging. We find that all variants of the method achieve a significant reduction in annotation cost, although their computational efficiency differs. In particular, the simplest variant, a two member committee with no parameters to tune, gives excellent results. We also show that sample selection yields a significant reduction in the size of the model used by the tagger.
An Application of a Runtime Epistemic Probabilistic Event Calculus to Decision-making in e-Health Systems
University of Verona(维罗纳大学);University of Naples Federico II(那不勒斯费德里科二世大学)
我们提出并讨论了一个运行时架构,集成了感官数据和分类器与基于逻辑的决策系统的背景下,电子健康系统的儿童神经运动障碍的康复。在该应用中,儿童以游戏的形式执行康复任务。该系统的主要目的是导出一组参数,该参数是儿童的认知和行为表现的当前水平(例如,参与、注意力、任务准确性)从可用的传感器和分类器(例如,眼睛跟踪器、运动传感器、情感识别技术)并相应地做出决定。这些决定通常旨在通过在孩子注意力低下时触发适当的重新参与刺激来提高孩子的表现,通过改变游戏或当孩子对任务失去兴趣时使其变得更加困难,因为它太容易了。除了最先进的技术,情绪识别和头部姿态估计,我们使用的概率和认知逻辑编程方言的事件演算,称为认知概率事件演算的运行时变体。特别地,该符号框架的概率组件允许与机器学习技术的自然接口。我们概述了架构及其组件,并通过一个运行的例子和实验的讨论,它的一些特点。《逻辑程序设计理论与实践》(Theory and Practice of Logic Programming,TPLP)
h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective
Research Inst. of Trustworthy Autonomous Systems & Dept. of Computer Science and Engineering, SUSTech, China(可信自主系统研究所及计算机科学与工程系,深圳技术大学,中国);Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Dept. of Computer Science and Engineering, SUSTech, and Peng Cheng Lab, China(脑启发智能计算广东省重点实验室、计算机科学与工程系,深圳技术大学,以及鹏城实验室,中国);Faculty of Engineering and Information Technology, University of Technology Sydney, Australia(工程与信息技术学院,悉尼大学,澳大利亚);Inst. of Biomedical Engineering, Dept. of Engineering Science, University of Oxford, UK(生物医学工程研究所、工程科学系,牛津大学,英国)