This paper describes the submissions of Huawei Translation Services Center(HW-TSC) to WMT24 chat translation shared task on English$\leftrightarrow$Germany (en-de) bidirection. The experiments involved fine-tuning models using chat data and exploring various strategies, including Minimum Bayesian Risk (MBR) decoding and self-training. The results show significant performance improvements in certain directions, with the MBR self-training method achieving the best results. The Large Language Model also discusses the challenges and potential avenues for further research in the field of chat translation.
Convergences and Divergences between Automatic Assessment and Human Evaluation: Insights from Comparing ChatGPT-Generated Translation and Neural Machine Translation
School of Foreign Languages, Shanghai Jiao Tong University（上海交通大学外国语学院）；Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系）；Institute of Corpus Studies and Applications, Shanghai International Studies University（上海外国语大学语料库研究与应用研究所）
University of California, Santa Barbara（加州大学圣巴巴拉分校）；Microsoft（微软）；NEC Laboratories America（NEC美国实验室）
Text summarization has been a crucial problem in natural language processing (NLP) for several decades. It aims to condense lengthy documents into shorter versions while retaining the most critical information. Various methods have been proposed for text summarization, including extractive and abstractive summarization. The emergence of large language models (LLMs) like GPT3 and ChatGPT has recently created significant interest in using these models for text summarization tasks. Recent studies \cite{goyal2022news, zhang2023benchmarking} have shown that LLMs-generated news summaries are already…
Large language models have recently attracted significant attention due to their impressive performance on a variety of tasks. ChatGPT developed by OpenAI is one such implementation of a large, pre-trained language model that has gained immense popularity among early adopters, where certain users go to the extent of characterizing it as a disruptive technology in many domains. Understanding such early adopters' sentiments is important because it can provide insights into the potential success or failure of the technology, as well as its strengths and weaknesses. In this paper, we conduct a mix…
Distinguishing Translations by Human, NMT, and ChatGPT: A Linguistic and Statistical Approach
School of Foreign Languages, Shanghai Jiao Tong University（上海交通大学外国语学院）；Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系）；Institute of Corpus Studies and Applications, Shanghai International Studies University（上海外国语大学语料库研究与应用研究院）
Hidden in Plain Sight: Exploring Chat History Tampering in Interactive Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Exploring the Limits of ChatGPT in Software Security Applications
University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；ASU（亚利桑那州立大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；University of Wisconsin, Madison（威斯康星大学麦迪逊分校）
What does ChatGPT return about human values? Exploring value bias in ChatGPT using a descriptive value theory
Institute D’Or for Research and Teaching（多尔研究与教学学院）；School of Psychology（心理学院）；School of Information Management（信息管理学院）；Dublin City University（都柏林城市大学）；University of Western Australia（西澳大学）
本研究基于 Schwartz 基础价值观理论，通过 OpenAI API 重复提示 ChatGPT 生成文本并使用词袋方法分析，发现其输出无明显价值偏见且高保真传递了价值内容，仅存在社会导向价值观的融合。
AI Ethics and Social Norms: Exploring ChatGPT's Capabilities From What to How
University of Toronto（多伦多大学）；Drexel University（德雷塞尔大学）；University of Siegen（锡根大学）；Westphal College of Media Arts and Design（媒体艺术与设计西弗尔学院）；Digital Media Department（数字媒体系）；Information Systems（信息系统）
ORIBA: Exploring LLM-Driven Role-Play Chatbot as a Creativity Support Tool for Original Character Artists
Computer Science Research Centre, Royal College of Art（皇家艺术学院计算机科学研究中心）；Digital Media, School of Literature, Media, and Communication, Georgia Institute of Technology（佐治亚理工学院数字媒体系）；Princeton University（普林斯顿大学）；Digital Media, Georgia Institute of Technology（佐治亚理工学院数字媒体系）；Division of Integrative Systems and Design, The Hong Kong University of Science and Technology（香港科学大学整合系统与设计 division）；Industrial Design Department, College of Engineering, KAIST（韩国科学技术院工程学院工业设计系）
ChatGPT as Data Augmentation for Compositional Generalization: A Case Study in Open Intent Detection
Ingenuity Labs Research Institute（创新实验室研究所）；Queen’s University（女王大学）；Smith School of Business（史密斯商学院）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Peng Cheng Laboratory（鹏城实验室）；Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室）
Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
The University of Georgia（佐治亚大学）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Northwestern Polytechnical University（西北工业大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Mayo Clinic（梅奥诊所）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
In this paper, we address the task of improving pair-wise machine translation for specific low resource Indian languages. Multilingual NMT models have demonstrated a reasonable amount of effectiveness on resource-poor languages. In this work, we show that the performance of these models can be significantly improved upon by using back-translation through a filtered back-translation process and subsequent fine-tuning on the limited pair-wise language corpora. The analysis in this paper suggests that this method can significantly improve a multilingual model's performance over its baseline, yiel…
AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial Topics
IMDEA Networks Institute（IMDEA网络研究所）；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；University of Surrey（萨里大学）；King’s College London（伦敦国王学院）；VRAIN, Universitat Politecnica de Valencia（瓦伦西亚理工大学VRAIN研究所）
Exploring a Large Language Model for Transforming Taxonomic Data into OWL: Lessons Learned and Implications for Ontology Development
University of Sao Paulo, Polytechnic School, Computer Engineering and Digital Systems（圣保罗大学，理工学院，计算机工程与数字系统）；University of Sao Paulo, Center for Artificial Intelligence (C4AI)（圣保罗大学，人工智能中心（C4AI））；University of Twente, Faculty of Electrical Engineering, Mathematics and Computer Science（代尔夫特理工大学，电气工程、数学与计算机科学学院）；Leiden University Medical Center, Human Genetics（莱顿大学医学中心，人类遗传学）；University of Sao Paulo, Institute of Mathematics and Computer Science（圣保罗大学，数学与计算机科学研究所）；University of Sao Paulo, Luiz de Queiroz College of Agriculture, Center for Advanced Studies on Applied Economics（圣保罗大学，路易斯·德·奎罗兹农业学院，应用经济学研究中心）；University of Sao Paulo, Institute of Mathematics and Statistics（圣保罗大学，数学与统计学研究所）；Embrapa Digital Agriculture, Campinas（巴西数字农业研究所，坎皮纳斯）
Large language models (LLMs) have recently taken the world by storm. They can generate coherent text, hold meaningful conversations, and be taught concepts and basic sets of instructions - such as the steps of an algorithm. In this context, we are interested in exploring the application of LLMs to graph drawing algorithms by performing experiments on ChatGPT. These algorithms are used to improve the readability of graph visualizations. The probabilistic nature of LLMs presents challenges to implementing algorithms correctly, but we believe that LLMs' ability to learn from vast amounts of data …
School of Computer Sciences, NISER（国家科学教育与研究学院计算机科学学院）；Homi Bhabha National Institute（霍米·巴巴国家研究所）
Neural Machine Translation (NMT) models have been effective on large bilingual datasets. However, the existing methods and techniques show that the model's performance is highly dependent on the number of examples in training data. For many languages, having such an amount of corpora is a far-fetched dream. Taking inspiration from monolingual speakers exploring new languages using bilingual dictionaries, we investigate the applicability of bilingual dictionaries for languages with extremely low, or no bilingual corpus. In this paper, we explore methods using bilingual dictionaries with an NMT …
Neural machine translation (NMT) systems have been shown to give undesirable translation when a small change is made in the source sentence. In this paper, we study the behaviour of NMT systems when multiple changes are made to the source sentence. In particular, we ask the following question "Is it possible for an NMT system to predict same translation even when multiple words in the source sentence have been replaced?". To this end, we propose a soft-attention based technique to make the aforementioned word replacements. The experiments are conducted on two language pairs: English-German (en…
Neural chat translation aims to translate bilingual conversational text, which has a broad application in international exchanges and cooperation. Despite the impressive performance of sentence-level and context-aware Neural Machine Translation (NMT), there still remain challenges to translate bilingual conversational text due to its inherent characteristics such as role preference, dialogue coherence, and translation consistency. In this paper, we aim to promote the translation quality of conversational text by modeling the above properties. Specifically, we design three latent variational mo…
