Data annotation is essential for supervised learning, yet producing accurate, unbiased, and scalable labels remains challenging as datasets grow in size and modality. Traditional human-centric pipelines are costly, slow, and prone to annotator variability, motivating reliability-aware automated annotation. We present AURA (Agentic AI for Unified Reliability Modeling and Annotation Aggregation), an agentic AI framework for large-scale, multi-modal data annotation. AURA coordinates multiple AI agents to generate and validate labels without requiring ground truth. At its core, AURA adapts a classical probabilistic model that jointly infers latent true labels and annotator reliability via confusion matrices, using Expectation-Maximization to reconcile conflicting annotations and aggregate noisy predictions. Across the four benchmark datasets evaluated, AURA achieves accuracy improvements of up to 5.8% over baseline. In more challenging settings with poor quality annotators, the improvement is up to 50% over baseline. AURA also accurately estimates the reliability of annotators, allowing assessment of annotator quality even without any pre-validation steps.
Beyond Manual Curation: Augmenting Targeted Protein Degradation Databases via Agentic Literature Extraction Workflows
Department of Computer Science and Engineering（计算机科学与工程系）；Chalmers University of Technology（楚姆勒斯技术大学）；University of Gothenburg（哥德堡大学）；Yardi School of Artificial Intelligence（Yardi人工智能学院）；Indian Institute of Technology Delhi（德里印度理工学院）
Modern self-driving autonomy systems heavily rely on deep learning. As a consequence, their performance is influenced significantly by the quality and richness of the training data. Data collecting platforms can generate many hours of raw data in a daily basis, however, it is not feasible to label everything. It is thus of key importance to have a mechanism to identify "what to label". Active learning approaches identify examples to label, but their interestingness is tied to a fixed model performing a particular task. These assumptions are not valid in self-driving, where we have to solve a d…
We currently do not have an understanding of semi-supervised learning (SSL) objectives such as pseudo-labelling and entropy minimization as log-likelihoods, which precludes the development of e.g. Bayesian SSL. Here, we note that benchmark image datasets such as CIFAR-10 are carefully curated, and we formulate SSL objectives as a log-likelihood in a generative model of data curation that was initially developed to explain the cold-posterior effect (Aitchison 2020). SSL methods, from entropy minimization and pseudo-labelling, to state-of-the-art techniques similar to FixMatch can be understood …
What Do Patients Say About Their Disease Symptoms? Deep Multilabel Text Classification With Human-in-the-Loop Curation for Automatic Labeling of Patient Self Reports of Problems
Automatic Dataset Construction (ADC): Sample Collection, Data Curation, and Beyond
University of California, Santa Cruz（加州大学圣克鲁兹分校）；Amazon（亚马逊）；SUSTech（华南理工大学）；Zhejiang University（浙江大学）；Yale University（耶鲁大学）；Carnegie Mellon University（卡内基梅隆大学）；HKUST (GZ)（香港科技大学）；Nanyang Technological University（南洋理工大学）；Microsoft（微软）
Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation
Beijing Institute of Technology（北京理工大学）；State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；DataCanvas；Beijing University of Posts and Telecommunications（北京邮电大学）；Shenzhen MSU-BIT University（深圳MSU-BIT大学）
Can Failed Trajectories Reveal Where Agents Go Wrong? Agentic Failure Attribution Without Step-Level Labels
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；Australian Artificial Intelligence Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究所）
A Modular, Data-Free Pipeline for Multi-Label Intention Recognition in Transportation Agentic AI Applications
Department of Infrastructure Engineering, Faculty of Engineering and Information Technology, The University of Melbourne, Australia（工程与信息技术学院基础设施工程系，墨尔本大学，澳大利亚）；School of Computing, National University of Singapore, Singapore（计算学院，新加坡国立大学，新加坡）；Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore（高性能计算研究所，科学、技术与研究局（A*STAR），新加坡）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR（土木与环境工程系，香港科学与技术大学，香港特别行政区）
Towards a Pseudo-Labeling Workflow for Celltype-Classification from Explanted Brain Slice Recordings
RWTH University Hospital, Department of Neurosurgery（RWTH大学医院神经外科部门）；RWTH University Hospital, Department of Epileptology（RWTH大学医院癫痫学部门）；University of Duisburg-Essen, Intelligent Embedded Systems Lab（杜伊斯堡-埃森大学智能嵌入式系统实验室）
Google Research（谷歌研究院）；University of Pennsylvania（宾夕法尼亚大学）
NLP researchers need more, higher-quality text datasets. Human-labeled datasets are expensive to collect, while datasets collected via automatic retrieval from the web such as WikiBio are noisy and can include undesired biases. Moreover, data sourced from the web is often included in datasets used to pretrain models, leading to inadvertent cross-contamination of training and test sets. In this work we introduce a novel method for efficient dataset curation: we use a large language model to provide seed generations to human raters, thereby changing dataset authoring from a writing task to an ed…
RTLCurator: Label-Efficient Data Curation for RTL Generation
University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）
Design and evaluation of an agentic workflow for crisis-related synthetic tweet datasets
Department of Computer Science, University College London, London, United Kingdom（伦敦大学学院计算机科学系，伦敦，英国）；National Institute of Informatics, Tokyo, Japan（日本信息学研究院，东京，日本）
Web-scraped, in-the-wild datasets have become the norm in face recognition research. The numbers of subjects and images acquired in web-scraped datasets are usually very large, with number of images on the millions scale. A variety of issues occur when collecting a dataset in-the-wild, including images with the wrong identity label, duplicate images, duplicate subjects and variation in quality. With the number of images being in the millions, a manual cleaning procedure is not feasible. But fully automated methods used to date result in a less-than-ideal level of clean dataset. We propose a se…
RoboGene: Diversity-Driven Agentic Task Generation Framework for VLA Pre-training
Beijing Innovation Center of Humanoid Robotics（北京人形机器人创新中心）；The School of Advanced Manufacturing and Robotics, Peking University（北京大学先进制造与机器人学院）；Beijing Institute of Technology（北京理工大学）；The School of Mechanical Engineering and Automation, Beihang University（北航机械工程与自动化学院）；State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学多媒体信息处理国家重点实验室）
Objective: The majority of detailed patient information in real-world data (RWD) is only consistently available in free-text clinical documents. Manual curation is expensive and time-consuming. Developing natural language processing (NLP) methods for structuring RWD is thus essential for scaling real-world evidence generation. Materials and Methods: Traditional rule-based systems are vulnerable to the prevalent linguistic variations and ambiguities in clinical text, and prior applications of machine-learning methods typically require sentence-level or report-level labeled examples that are har…
Multistep Automated Data Labelling Procedure (MADLaP) for Thyroid Nodules on Ultrasound: An Artificial Intelligence Approach for Automating Image Annotation
Duke University（杜克大学）；Duke University Medical Center（杜克大学医学中心）
As the adoption of deep learning techniques in industrial applications grows with increasing speed and scale, successful deployment of deep learning models often hinges on the availability, volume, and quality of annotated data. In this paper, we tackle the problems of efficient data labeling and annotation verification under the human-in-the-loop setting. We showcase that the latest advancements in the field of self-supervised visual representation learning can lead to tools and methods that benefit the curation and engineering of natural image datasets, reducing annotation cost and increasin…
SCOPE-MRI: Bankart Lesion Detection as a Case Study in Data Curation and Deep Learning for Challenging Diagnoses
Pritzker School of Medicine, University of Chicago（芝加哥大学普ritzker医学学院）；University of Chicago（芝加哥大学）；Department of Computer Science, University of Chicago（芝加哥大学计算机科学系）；Department of Orthopaedic Surgery & Rehabilitation Medicine, UChicago Medicine（芝加哥医学中心骨科与康复医学系）
Continual learning (CL) is widely regarded as crucial challenge for lifelong AI. However, existing CL benchmarks, e.g. Permuted-MNIST and Split-CIFAR, make use of artificial temporal variation and do not align with or generalize to the real-world. In this paper, we introduce CLEAR, the first continual image classification benchmark dataset with a natural temporal evolution of visual concepts in the real world that spans a decade (2004-2014). We build CLEAR from existing large-scale image collections (YFCC100M) through a novel and scalable low-cost approach to visio-linguistic dataset curation.…
