The development of Large Language Models (LLMs) in various languages has been advancing, but the combination of non-English languages with domain-specific contexts remains underexplored. This paper presents our findings from training and evaluating a Japanese business domain-specific LLM designed to better understand business-related documents, such as the news on current affairs, technical reports, and patents. Additionally, LLMs in this domain require regular updates to incorporate the most recent knowledge. Therefore, we also report our findings from the first experiments and evaluations involving updates to this LLM using the latest article data, which is an important problem setting that has not been addressed in previous research. From our experiments on a newly created benchmark dataset for question answering in the target domain, we found that (1) our pretrained model improves QA accuracy without losing general knowledge, and (2) a proper mixture of the latest and older texts in the training data for the update is necessary. Our pretrained model and business domain benchmark are publicly available to support further studies.
SimInterview: Transforming Business Education through Large Language Model-Based Simulated Multilingual Interview Training System
Analytics Everywhere Lab, University of New Brunswick, Canada(新不伦瑞克大学分析 everywhere 实验室);University of Foreign Language Studies, University of Danang, Vietnam(越南丹绒大学外语学院);Faculty of Information Technology, University of Science, VNU-HCM, Vietnam(越南胡志明市大学信息科技学院);Faculty of Economics and Accounting, Quy Nhon University, Vietnam(越南奎隆大学经济与会计学院);Faculty of Natural Sciences, Quy Nhon University, Vietnam(越南奎隆大学自然科学学院)
Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs
Unbabel;Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico & Universidade de Lisboa (Lisbon ELLIS Unit)(里斯本大学技术学院及里斯本大学(里斯本ELLIS单位));MICS, CentraleSupélec, Université Paris-Saclay(MICS,CentraleSupélec,巴黎萨克雷大学)
HumanLLM: Towards Personalized Understanding and Simulation of Human Nature
University of Science and Technology of China(科学技术大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Microsoft Research Asia(微软亚洲研究院)
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
Babelscape(巴别景观公司);LAION(LAION组织);TU Darmstadt(达姆施塔特工业大学);UNC Chapel-Hill(北卡罗来纳大学教堂山分校);Detomo Inc.(Detomo公司);CSIRO’s Data61(联邦科学与工业研究组织Data61);Monash University(莫纳什大学);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Carnegie Mellon University(卡内基梅隆大学);Allen Institute for AI(艾伦人工智能研究所);DeepTensor AB(DeepTensor AB公司);WASP Media & Language(WASP媒体与语言机构);Umeå University(于默奥大学);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);TU Wien(维也纳技术大学);Institute of Science Tokyo(东京科学大学);Hudson Labs(哈德逊实验室);Luleå University of Technology(吕勒奥理工大学);ETH Zurich(苏黎世联邦理工学院);RunPod(RunPod公司);OpenChat(OpenChat机构);RIKEN CCS(理化学研究所计算科学中心);Chattermill AI(Chattermill人工智能公司);ASC27(ASC27机构);University of Massachuset(马萨诸塞大学)