Recently, the development of open-source large language models (LLMs) has advanced rapidly. Nevertheless, due to data constraints, the capabilities of most open-source LLMs are primarily focused on English. To address this issue, we introduce the concept of $\textit{chat vector}$ to equip pre-trained language models with instruction following and human value alignment via simple model arithmetic. The chat vector is derived by subtracting the weights of a pre-trained base model (e.g. LLaMA2) from those of its corresponding chat model (e.g. LLaMA2-chat). By simply adding the chat vector to a continual pre-trained model's weights, we can endow the model with chat capabilities in new languages without the need for further training. Our empirical studies demonstrate the superior efficacy of the chat vector from three different aspects: instruction following, toxicity mitigation, and multi-turn dialogue. Moreover, to showcase the adaptability of our approach, we extend our experiments to encompass various languages, base models, and chat vectors. The results underscore the chat vector's simplicity, effectiveness, and wide applicability, making it a compelling solution for efficiently enabling conversational capabilities in pre-trained language models. Our code is available at https://github.com/aqweteddy/ChatVector.
ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
Tencent AI Lab（腾讯人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）；Shanghai Jiao Tong University（上海交通大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）
Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant
Czech Technical University in Prague（布拉格捷克技术大学）；National Institute of Mental Health in Prague（布拉格国家精神卫生研究所）；The University of Hong Kong（香港大学）；Shenzhen Mental Health Centre（深圳市精神卫生中心）
FreezeEmpath: Efficient Training for Empathetic Spoken Chatbots with Frozen LLMs
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（智能信息处理重点实验室，计算技术研究所，中国科学院）；State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）
An Empirical Study of Instruction-tuning Large Language Models in Chinese
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；APUS AiLMe Lab（APUS AiLMe实验室）
ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Université de Montréal（蒙特利尔大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Waseda University（早稻田大学）
ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models
University of Chinese Academy of Sciences（中国科学院大学）；Chinese Information Processing Laboratory（中文信息处理实验室）；Ricoh Software Research Center (Beijing) Co., Ltd.（理光软件研究中心（北京）有限公司）
From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning
USTC（University of Science and Technology of China）；Alibaba Group（阿里巴巴集团）；CASIA（Chinese Academy of Sciences Institute of Automation）；SIAT（Institute of Automation, Chinese Academy of Sciences）
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Bytedance Inc.（字节跳动公司）
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
York University（约克大学）；Nanyang Technological University（南洋理工大学）；Dialpad Canada Inc.（Dialpad加拿大公司）；Royal Bank of Canada（加拿大皇家银行）；Salesforce Research（赛富时研究院）
Chinese Open Instruction Generalist: A Preliminary Release
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；The University of Sheffield（谢菲尔德大学）；University of Michigan Ann Arbor（密歇根大学安娜堡分校）；Dartmouth University（达特茅斯学院）；Zhejiang University（浙江大学）；Beihang University（北京航空航天大学）；Carnegie Mellon University（卡内基梅隆大学）
Chat-TS: Enhancing Multi-Modal Reasoning Over Time-Series and Natural Language Data
Electrical and Computer Engineering, Queen’s University（皇后大学电气与计算机工程学院）；Mechanical and Materials Engineering, Queen’s University（皇后大学机械与材料工程学院）；Ingenuity Labs Research Institute, Queen’s University（皇后大学创新实验室研究 institute）
Demystifying Instruction Mixing for Fine-tuning Large Language Models
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；LibrAI；Monash University（莫纳什大学）；University of British Columbia（不列颠哥伦比亚大学）；The University of Melbourne（墨尔本大学）
