Many language generation models are now available for a wide range of generation tasks, including machine translation and summarization. Combining such diverse models may lead to further progress, but ensembling generation models is challenging during inference: conventional ensembling methods (e.g., shallow fusion) require that the models share vocabulary/tokenization schemes. We introduce Twist decoding, a simple and general text generation algorithm that benefits from diverse models at inference time. Our method does not assume the vocabulary, tokenization or even generation order is shared. Our extensive evaluations on machine translation and scientific paper summarization demonstrate that Twist decoding substantially outperforms each model decoded in isolation over various scenarios, including cases where domain-specific and general-purpose models are both available. Twist decoding also consistently outperforms the popular reranking heuristic where output candidates from one model are rescored by another. We hope that our work will encourage researchers and practitioners to examine generation models collectively, not just independently, and to seek out models with complementary strengths to the currently available models. Our code is available at https://github.com/jungokasai/twist_decoding.
CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generation
School of Computation, Information, and Technology, Technical University of Munich(计算、信息与技术学院,慕尼黑技术大学);Graduate Institute of International and Development Studies, Geneva(国际与发展研究研究生院,日内瓦);JPMorgan AI Research(摩根大通人工智能研究)
Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding
The Hong Kong Polytechnic University(香港理工大学);IROOTECH TECHNOLOGY(IROOTECH技术公司);Wolf 1069 b Lab, Sany Group(三一集团沃尔夫1069实验室);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Zhejiang University(浙江大学)
DECIDER: A Dual-System Rule-Controllable Decoding Framework for Language Generation
Key Laboratory of Brain Health Intelligent Evaluation and Intervention, Ministry of Education, School of Medical Technology, Bejing Institute of Technology (BIT), China(脑健康智能评估与干预关键实验室,教育部,医学技术学院,北京理工大学(BIT),中国);School of Computer Science and Technology, BIT, China(计算机科学与技术学院,北京理工大学(BIT),中国);Nanjing University of Aeronautics and Astronautics, Nanjing, China(南京航空航天大学,南京,中国)