arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.07404cs.ROcs.AI
收藏

面向视觉语言动作微调的自适应容量分配

Adaptive Capacity Allocation for Vision Language Action Fine-tuning

Department of Electrical and Computer Engineering, Seoul National University(电气与计算机工程系,首尔国立大学);Department of Computer Science and Engineering, Sungkyunkwan University(计算机科学与工程系,全北国立大学)

Donghoon Kim, Minji Bae, Unghui Nam, Gyeonghun Kim, Suyun Lee, Kyuhong Shim, Byonghyo Shim

AI总结:

本文提出LoRA-SP方法,通过自适应秩微调提升视觉语言动作模型在多任务场景中的性能和泛化能力。

摘要:

视觉语言动作模型(VLAs)越来越多地用于物理人工智能,但将预训练的VLA模型部署到未见的环境、具象或任务中仍需要适应。参数高效微调(PEFT),特别是LoRA,是VLA策略常用的微调方法,然而暴露的容量调节器,即秩,并不均匀转移:机器人转移表现出比语言微调更高的固有秩且任务相关性更强。小秩足以满足LLM(例如,r∈{4,8}),而谱分析表明VLAs可能需要远更大的秩(例如,r≈128)或接近全秩,这种不匹配在多任务设置中会加剧。我们提出了LoRA-SP(选择-修剪),一种自适应秩微调方法,它用输入和层级的容量替代固定秩更新。LoRA-SP采用类似SVD的参数化方法,使用一个小的路由器,其非负分数作为共享向量库的奇异值。通过累积平方分数的能耗目标E(k)≥η选择活跃集,提供了一种直接连接近似误差的途径。在训练过程中,η将能量集中在少数方向上,并教导路由器依赖更少的向量同时保持准确性。这产生了紧凑的适配器,减少了跨任务干扰并提高了泛化能力。在四个收集于未见AgileX PiPER机械臂的真实机器人操作任务上,针对两种VLA骨干网络(π₀和SmolVLA),LoRA-SP在远少于全微调的可训练参数下实现了匹配或超越,且在标准LoRA基础上,多任务成功率提高了最高31.6%。

英文摘要:

Vision language action models (VLAs) are increasingly used for Physical AI, but deploying a pre-trained VLA model to unseen environments, embodiments, or tasks still requires adaptation. Parameter-efficient fine-tuning (PEFT), especially LoRA, is common for VLA policies, yet the exposed capacity knob, the rank, does not transfer uniformly: robotics transfer exhibits a higher and task-varying intrinsic rank than language fine-tuning. Small ranks suffice for LLMs (e.g., $r \in \{4, 8\}$), while spectral analyses indicate VLAs may require much larger ranks (e.g., $r \approx 128$) or near-full rank, a mismatch that worsens in multi-task settings. We present LoRA-SP (Select-Prune), a rank-adaptive fine-tuning method that replaces fixed-rank updates with input- and layer-wise capacity. LoRA-SP uses an SVD-style parameterization with a small router whose nonnegative scores act as singular values over a shared vector bank. The active set is chosen by an energy target on the cumulative squared scores $E(k) \ge η$, providing a direct link to approximation error via our spectral analysis. During training, $η$ concentrates energy on a few directions and teaches the router to rely on fewer vectors while preserving accuracy. This yields compact adapters that reduce cross-task interference and improve generalization. On four real-robot manipulation tasks collected on an unseen AgileX PiPER arm, across two VLA backbones ($π_0$ and SmolVLA), LoRA-SP matches or exceeds full fine-tuning with far fewer trainable parameters, and improves multi-task success by up to 31.6% over standard LoRA while remaining robust to rank choice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑