arXiv 2201.082782022-01-20cs.AIcs.LG

Lifelong Learning Metrics

Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）

Alexander New, Megan Baker, Eric Nguyen, Gautam Vallabha

英文摘要：

The DARPA Lifelong Learning Machines (L2M) program seeks to yield advances in artificial intelligence (AI) systems so that they are capable of learning (and improving) continuously, leveraging data on one task to improve performance on another, and doing so in a computationally sustainable way. Performers on this program developed systems capable of performing a diverse range of functions, including autonomous driving, real-time strategy, and drone simulation. These systems featured a diverse range of characteristics (e.g., task structure, lifetime duration), and an immediate challenge faced by the program's testing and evaluation team was measuring system performance across these different settings. This document, developed in close collaboration with DARPA and the program performers, outlines a formalism for constructing and characterizing the performance of agents performing lifelong learning scenarios.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Latent Properties of Lifelong Learning Systems

Intelligent Systems Center（智能系统中心）；Johns Hopkins Applied Physics Lab（约翰斯·霍普金斯应用物理实验室）

Creating artificial intelligence (AI) systems capable of demonstrating lifelong learning is a fundamental challenge, and many approaches and metrics have been proposed to analyze algorithmic properties. However, for existing lifelong learning metrics, algorithmic contributions are confounded by task and scenario structure. To mitigate this issue, we introduce an algorithm-agnostic explainable surrogate-modeling approach to estimate latent properties of lifelong learning algorithms. We validate the approach for estimating these properties via experiments on synthetic data. To validate the struc…

2022-07-28cs.AIcs.LG

An Introduction to Lifelong Supervised Learning

FAIR, Meta AI（元人工智能基础人工智能研究院）；Universite de Montreal（蒙特利尔大学）；Carnegie Mellon University（卡内基梅隆大学）；Polytechnique Montréal（蒙特利尔理工学院）；Quebec Artificial Intelligence Institute (Mila)（魁北克人工智能研究院（Mila））

This primer is an attempt to provide a detailed summary of the different facets of lifelong learning. We start with Chapter 2 which provides a high-level overview of lifelong learning systems. In this chapter, we discuss prominent scenarios in lifelong learning (Section 2.4), provide 8 Introduction a high-level organization of different lifelong learning approaches (Section 2.5), enumerate the desiderata for an ideal lifelong learning system (Section 2.6), discuss how lifelong learning is related to other learning paradigms (Section 2.7), describe common metrics used to evaluate lifelong learn…

2022-07-12cs.AIcs.LG

通过相似性启发式终身提示调优缓解负迁移

Mitigate Negative Transfer with Similarity Heuristic Lifelong Prompt Tuning

University of Science and Technology of China（中国科学技术大学）

针对终身提示调优中不相似任务易引发负迁移的问题，提出带可学习相似性度量和参数池的SHLPT框架，在基准测试中性能优于现有最优方法且抗负迁移鲁棒性强。

2024-06-18cs.AIcs.CLcs.LG

Enabling Reproducibility and Meta-learning Through a Lifelong Database of Experiments (LDE)

IBM Research（IBM研究院）；University of Groningen（格罗宁根大学）

Artificial Intelligence (AI) development is inherently iterative and experimental. Over the course of normal development, especially with the advent of automated AI, hundreds or thousands of experiments are generated and are often lost or never examined again. There is a lost opportunity to document these experiments and learn from them at scale, but the complexity of tracking and reproducing these experiments is often prohibitive to data scientists. We present the Lifelong Database of Experiments (LDE) that automatically extracts and stores linked metadata from experiment artifacts and provid…

2022-02-23cs.AIcs.LG

终身学习AI加速器的设计原则

Design Principles for Lifelong Learning AI Accelerators

University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；Sandia National Laboratories（桑迪亚国家实验室）；Argonne National Laboratory（阿贡国家实验室）；Rochester Institute of Technology（罗切斯特理工学院）；Forschungszentrum Jülich（于利希研究中心）；RWTH Aachen（亚琛工业大学）；Tenstorrent Inc.（Tenstorrent公司）；University of Pisa（比萨大学）；Air Force Research Laboratory（美国空军研究实验室）；ECS Federal（ECS联邦公司）

针对终身学习AI落地边缘平台的硬件需求，本文明确了终身学习AI加速器的核心能力与评估指标，梳理现有边缘AI加速器情况，并结合新兴技术探索其未来设计方向。

2023-10-05cs.AIcs.LGcs.SY

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Johns Hopkins University Applied Physics Laboratory（约翰斯·霍普金斯大学应用物理实验室）；Teledyne Scientific Company（Teledyne科学公司）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Southern California（南加州大学）；Loughborough University（拉夫堡大学）；University of Michigan（密歇根大学）；SRI International（SRI国际）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Sandia National Laboratories（桑迪亚国家实验室）

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typica…

2023-01-18cs.CVcs.LGcs.RO

更多相关论文收起更多论文

通过动态知识空间实现自动驾驶的去混淆终身学习

Deconfounded Lifelong Learning for Autonomous Driving via Dynamic Knowledge Spaces

Tongji University（同济大学）

本文提出DeLL框架，结合DPMM与因果推理的front-door调整机制，构建动态知识空间以解决自动驾驶中的灾难性遗忘和知识迁移问题，并引入进化轨迹解码器提升规划能力。

2026-07-02cs.AIcs.LGcs.RO

CL2R: Compatible Lifelong Learning Representations

University of Florence（佛罗伦萨大学）

In this paper, we propose a method to partially mimic natural intelligence for the problem of lifelong learning representations that are compatible. We take the perspective of a learning agent that is interested in recognizing object instances in an open dynamic universe in a way in which any update to its internal feature representation does not render the features in the gallery unusable for visual search. We refer to this learning problem as Compatible Lifelong Learning Representations (CL2R) as it considers compatible representation learning within the lifelong learning paradigm. We identi…

2022-11-16cs.CVcs.LG

终身测试时适应的概率框架

A Probabilistic Framework for Lifelong Test-Time Adaptation

Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）

本文提出PETAL框架解决终身测试时适应问题，通过结合学生-教师模型与源模型正则化，并引入数据驱动的参数恢复技术防止模型漂移，在多个基准上取得优于现有方法的预测与不确定性估计表现。

2023-04-04cs.AIcs.CVcs.LG

Efficient Lifelong Learning with A-GEM

University of Oxford（牛津大学）；Facebook AI Research（脸书人工智能研究院）

In lifelong learning, the learner is presented with a sequence of tasks, incrementally building a data-driven prior which may be leveraged to speed up learning of a new task. In this work, we investigate the efficiency of current lifelong approaches, in terms of sample complexity, computational and memory cost. Towards this end, we first introduce a new and a more realistic evaluation protocol, whereby learners observe each example only once and hyper-parameter selection is done on a small and disjoint set of tasks, which is not used for the actual learning experience and evaluation. Second, w…

2019-01-09cs.LGstat.ML

终身度量学习

Lifelong Metric Learning

Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Rochester（罗切斯特大学）；University of Arkansas at Little Rock（阿肯色大学小石城分校）

最先进的在线学习方法只能学习预定义任务的度量。在本文中，我们认为终身学习问题模仿“人类学习”，即，赋予从新的在线样本学习到的用于新任务的度量新的能力，并结合先前的经验和知识。因此，我们提出了一个新的度量学习框架：终身度量学习（LML），它只利用新任务的数据来训练度量模型，同时保持原有的能力。更具体地说，建议的LML维护一个公共子空间的所有学习的指标，命名为终身字典，从公共子空间的知识转移到每个新的度量任务与任务特定的特质，并重新定义公共子空间随着时间的推移，以最大限度地提高所有度量任务的性能。对于模型优化，我们采用在线被动积极的优化算法来解决所提出的LML框架，终身字典和任务特定的分区交替和连续优化。最后，我们通过分析几个多任务度量学习数据集来评估我们的方法。大量的实验结果表明，所提出的框架的有效性和效率。

2017-06-12cs.AIcs.LG

LeanAgent：用于形式化定理证明的终身学习

LeanAgent: Lifelong Learning for Formal Theorem Proving

California Institute of Technology（加州理工学院）；Stanford University（斯坦福大学）；University of Wisconsin, Madison（威斯康星大学麦迪逊分校）

针对形式化定理证明中静态LLM泛化性差的问题，提出终身学习框架LeanAgent，通过课程学习、动态数据库和渐进训练持续扩展数学知识，成功证明155个高级定理，并在稳定性和后向迁移上表现优越。

2025-03-06cs.AIcs.LGcs.LO

终身学习的持续评估：识别稳定性间隙

Continual evaluation for lifelong learning: Identifying the stability gap

KU Leuven（鲁汶大学）

本文针对持续学习模型在开始学习新任务时出现的暂时性遗忘现象（即“稳定性间隙”），提出了基于每次迭代评估的持续评估框架和新指标，揭示了多种主流方法均存在该问题，并给出了概念性解释。

2023-03-30cs.AIcs.CVcs.LG

CoReLIN: 基于约束推理的零样本终身交互式导航

CoReLIN: Constraint-based Reasoning for Zero-shot Lifelong Interactive Navigation

Purdue University（普渡大学）；Indian Institute of Technology, Kharagpur（印度理工学院卡哈拉格普尔分校）；DEVCOM Army Research Lab（DEVCOM陆军研究实验室）；National University of Singapore（新加坡国立大学）

提出CoReLIN框架，利用LLM和场景图推理，在终身交互式导航中通过移动物体开辟路径并完成任务，显著提升长期效率。

2026-07-01cs.AIcs.CVcs.RO