arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2210.14161cs.CVcs.AI

Aligning MAGMA by Few-Shot Learning and Finetuning

Polytechnique Montreal(蒙特利尔理工学院);Universite de Montreal(蒙特利尔大学);MILA(蒙特利尔学习算法研究所)

Jean-Charles Layoun, Alexis Roger, Irina Rish

英文摘要:

The goal of vision-language modeling is to allow models to tie language understanding with visual inputs. The aim of this paper is to evaluate and align the Visual Language Model (VLM) called Multimodal Augmentation of Generative Models through Adapter-based finetuning (MAGMA) with human values. MAGMA is a VLM that is capable of image captioning and visual question-answering. We will evaluate its alignment in three different scenarios. To begin, we assess MAGMA's out-of-the-box alignment through the checkpoint provided by Hugging Face. Then, we measure if few-shot learning manages to improve the results. Finally, we finetune the model on aligned examples and evaluate its behavior.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑