HKUST(GZ)(香港科技大学(广州));University of British Columbia(不列颠哥伦比亚大学);University of Tübingen(蒂宾根大学);Harbin Institute of Technology(哈尔滨工业大学);Beijing Institute of Mathematical Sciences and Applications(北京数学科学与应用研究院);VecML Inc.(VecML公司)
Zeke Xie, Zheng He, Nan Lu, Lichen Bai, Bao Li, Shuo Yang, Mingming Sun, Ping Li
针对硬标签模糊数据导致模型过度自信的问题,提出 QLL 框架与 CPU 风险估计器,并通过混合生成数据验证其泛化提升。
摘要:
现实世界的数据通常包含内在的模糊性,而常见的单一硬标签标注范式忽视了这一点。使用带有这些硬标签的模糊数据进行标准训练,可能会产生过度自信的模型,从而导致泛化能力较差。在本文中,我们提出了一个名为 Quantized Label Learning(QLL,量化标签学习)的新颖框架来缓解这一问题。首先,我们将 QLL 表述为从带有硬标签的(非常)模糊的数据中学习:理想情况下,每个模糊实例都应关联一个真实软标签分布,用于描述其在每个类别中对应的概率权重;然而,这通常无法获得;在实践中,我们只能观察到每个实例的一个量化标签,即从相应的真实软标签分布中采样(量化)得到的硬标签,这可以看作是对真实软标签的有偏近似。其次,我们提出了一种 Class-wise Positive-Unlabeled(CPU,类别的正例-未标注)风险估计器,使我们能够仅从带有量化标签的模糊数据中训练准确的分类器。第三,为了模拟现实世界中带有量化标签的模糊数据集,我们设计了一种基于混合的模糊数据生成过程用于实证评估。实验表明,我们的 CPU 方法能够显著提升模型的泛化性能,并优于基线方法。
英文摘要:
Real-world data often contains intrinsic ambiguity that the common single-hard-label annotation paradigm ignores. Standard training using ambiguous data with these hard labels may produce overly confident models and thus leading to poor generalization. In this paper, we propose a novel framework called Quantized Label Learning (QLL) to alleviate this issue. First, we formulate QLL as learning from (very) ambiguous data with hard labels: ideally, each ambiguous instance should be associated with a ground-truth soft-label distribution describing its corresponding probabilistic weight in each class, however, this is usually not accessible; in practice, we can only observe a quantized label, i.e., a hard label sampled (quantized) from the corresponding ground-truth soft-label distribution, of each instance, which can be seen as a biased approximation of the ground-truth soft-label. Second, we propose a Class-wise Positive-Unlabeled (CPU) risk estimator that allows us to train accurate classifiers from only ambiguous data with quantized labels. Third, to simulate ambiguous datasets with quantized labels in the real world, we design a mixing-based ambiguous data generation procedure for empirical evaluation. Experiments demonstrate that our CPU method can significantly improve model generalization performance and outperform the baselines.
Information-theoretic Classification Accuracy: A Criterion that Guides Data-driven Combination of Ambiguous Outcome Labels in Multi-class Classification
Academy of Mathematics and Systems Science(数学与系统科学研究院);Chinese Academy of Sciences(中国科学院);School of Mathematical Sciences(数学科学学院);University of Chinese Academy of Sciences(中国科学院大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)
Diffusion Disambiguation Models for Partial Label Learning
College of Computer Science and Technology, Qingdao University(青岛大学计算机科学与技术学院);Department of Control Science and Engineering, College of Electronics and Information Engineering, Tongji University(电子信息工程学院控制科学与工程系,同济大学);School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院)
CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries
Beijing Key Laboratory of Embodied Intelligence Systems, Department of Automation, Tsinghua University, Beijing, China(北京智能体智能系统重点实验室,自动化系,清华大学,北京,中国);The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China(复杂系统认知与决策智能重点实验室,自动化研究所,中国科学院,北京,中国)
Towards Robust Incremental Learning under Ambiguous Supervision
School of Software Technology, Zhejiang University(浙江大学软件学院);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新区(滨江)区块链与数据安全研究院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院)
Multi-Label Adaptive Batch Selection by Highlighting Hard and Imbalanced Samples
Key Laboratory of Data Engineering and Visual Computing, Chongqing University of Posts and Telecommunications(重庆邮电大学数据工程与视觉计算重点实验室);School of Informatics, Aristotle University of Thessaloniki(塞萨洛尼基亚里士多德大学信息学院)
Active Learning with Imperfect Labels: Optimal Labeler Assignment and Sample Selection
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA(工业与系统工程学院 佐治亚理工学院 阿拉巴马州亚特兰大 30332, 美国);Ford Motor Company(福特汽车公司)
Towards Eliminating Hard Label Constraints in Gradient Inversion Attacks
School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所)