The current modus operandi in NLP involves downloading and fine-tuning pre-trained models consisting of millions or billions of parameters. Storing and sharing such large trained models is expensive, slow, and time-consuming, which impedes progress towards more general and versatile NLP methods that learn from and for many tasks. Adapters -- small learnt bottleneck layers inserted within each layer of a pre-trained model -- ameliorate this issue by avoiding full fine-tuning of the entire model. However, sharing and integrating adapter layers is not straightforward. We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages. The framework, built on top of the popular HuggingFace Transformers library, enables extremely easy and quick adaptations of state-of-the-art pre-trained models (e.g., BERT, RoBERTa, XLM-R) across tasks and languages. Downloading, sharing, and training adapters is as seamless as possible using minimal changes to the training scripts and a specialized infrastructure. Our framework enables scalable and easy access to sharing of task-specific models, particularly in low-resource scenarios. AdapterHub includes all recent adapter architectures and can be found at https://AdapterHub.ml.
Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model
York University（约克大学）
信息可视化（如条形图和折线图）在探索数据和传达见解方面非常受欢迎。对某些人来说，解释和理解这种可视化可能具有挑战性，例如那些视力受损或视觉素养低的人。在这项工作中，我们引入了一个新的数据集，并提出了一个神经模型自动生成图表的自然语言摘要。生成的摘要提供了对图表的解释，并传达了在该图表中发现的关键见解。我们的神经模型是通过扩展数据到文本生成任务的最先进模型开发的，该模型利用了基于transformer的编码器-解码器架构。我们发现，我们的方法在内容选择指标上远远优于基础模型（55.42% vs. 8.49%），并生成了更多信息，简洁和连贯的摘要。
Adapting Natural Language Processing Models Across Jurisdictions: A pilot Study in Canadian Cancer Registries
School of Population and Public Health University of British Columbia（流行病学与公共卫生学院 首都大学）；Newfoundland & Labrador Health Services（纽芬兰与拉布拉多省卫生服务）；Data Science Institute University of British Columbia（数据科学研究所 首都大学）
PsychAdapter: Adapting LLM Transformers to Reflect Traits, Personality and Mental Health
Stony Brook University（石溪大学）；New York University（纽约大学）；University of Melbourne（墨尔本大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Pennsylvania（宾夕法尼亚大学）；Stanford University（斯坦福大学）
FL-Tuning: Layer Tuning for Feed-Forward Network in Transformer
East China University of Science and Technology（华东理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Fudan University（复旦大学）；Tencent AI Lab（腾讯人工智能实验室）
