arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.03356cs.LGstat.AP
收藏

Pl@ntNet人工智能算法的协作学习:它是如何工作的以及我们如何改进它?

Cooperative learning of Pl@ntNet's Artificial Intelligence algorithm: how does it work and how can we improve it?

Univ. Montpellier(蒙彼利埃大学);CNRS(法国国家科学研究中心);IMAG(蒙彼利埃计算机科学与应用数学研究所);Inria(法国国家信息与自动化研究所);LIRMM(蒙彼利埃电子与计算机科学实验室);CIRAD(法国农业国际合作研究发展中心);AMAP(植物与微生物环境适应数学、信息与物理模型实验室);Institut Universitaire de France (IUF)(法国大学研究院)

Tanguy Lefort, Antoine Affouard, Benjamin Charlier, Jean-Christophe Lombardo, Mathias Chouet, Hervé Goëau, Joseph Salmon, Pierre Bonnet, Alexis Joly

AI总结:

针对Pl@ntNet植物识别系统因用户技能差异导致的标签噪声问题,提出一种基于用户专业度信任分数递归估计的标签聚合策略,有效过滤不可靠数据并提升AI模型性能。

摘要:

用于植物物种识别的深度学习模型依赖于大型标注数据集。PlantNet系统通过允许用户上传和标注植物观测数据来实现全球数据收集,但由于用户技能的多样性,这导致了标签噪声。达成共识对于训练至关重要,但收集数据的庞大规模使得传统的标签聚合策略面临挑战。现有方法要么保留所有观测数据,导致训练数据存在噪声;要么有选择地保留那些获得足够票数的数据,从而丢弃有价值的信息。此外,由于许多物种很少被观察到,用户的专业知识无法通过用户间的一致性来评估:否则,在AI训练步骤中,植物学专家的权重将低于普通用户。我们提出的标签聚合策略旨在协作训练植物识别AI模型。该策略基于用户从众包数据中识别植物物种的能力,将用户的专业知识估计为每个用户的信任分数。该信任分数根据当前估计的标签,从正确识别的物种中递归估计得出。这一可解释的分数利用了植物学专家的知识和用户的异质性。随后,我们的策略移除了不可靠的观测数据,但保留了具有有限可信标注的数据,这与其他方法不同。我们在PlantNet数据库的一个已发布的大型子集上评估了PlantNet的策略,该子集专注于欧洲植物群,包含超过6M的观测数据和800K用户。我们证明,基于用户专业知识的多样性来评估用户技能可以提升标注性能。我们的发现强调了人类标注与数据过滤在改进AI性能以获得精炼数据集方面的协同作用。我们探索了在人类输入之外结合基于AI的投票。这可以进一步增强人机交互,以检测不可靠的观测数据。

英文摘要:

Deep learning models for plant species identification rely on large annotated datasets. The PlantNet system enables global data collection by allowing users to upload and annotate plant observations, leading to noisy labels due to diverse user skills. Achieving consensus is crucial for training, but the vast scale of collected data makes traditional label aggregation strategies challenging. Existing methods either retain all observations, resulting in noisy training data or selectively keep those with sufficient votes, discarding valuable information. Additionally, as many species are rarely observed, user expertise can not be evaluated as an inter-user agreement: otherwise, botanical experts would have a lower weight in the AI training step than the average user. Our proposed label aggregation strategy aims to cooperatively train plant identification AI models. This strategy estimates user expertise as a trust score per user based on their ability to identify plant species from crowdsourced data. The trust score is recursively estimated from correctly identified species given the current estimated labels. This interpretable score exploits botanical experts' knowledge and the heterogeneity of users. Subsequently, our strategy removes unreliable observations but retains those with limited trusted annotations, unlike other approaches. We evaluate PlantNet's strategy on a released large subset of the PlantNet database focused on European flora, comprising over 6M observations and 800K users. We demonstrate that estimating users' skills based on the diversity of their expertise enhances labeling performance. Our findings emphasize the synergy of human annotation and data filtering in improving AI performance for a refined dataset. We explore incorporating AI-based votes alongside human input. This can further enhance human-AI interactions to detect unreliable observations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑