The capability to generate diverse text is a key challenge facing large language models (LLMs). Thus far, diversity has been studied via metrics such as $n$-gram diversity or diversity of BERT embeddings. However, for these kinds of diversity, the user has little control over the dimensions along which diversity is considered. For example, in the poetry domain, one might desire diversity in terms of rhyme and meter, whereas in the code domain, one might desire diversity in terms of the kinds of expressions used to solve a problem. We propose a diversity metric called structural diversity, where the user provides a mapping from generated text to features capturing the kinds of diversity that they care about. In addition, we propose a novel strategy called chain-of-specification (CoS) prompting for improving diversity by first having the LLM generate a specification encoding one instance of structural features, and then prompting the LLM to generate text that satisfies these features; notably, our strategy works with blackbox LLMs. In our experiments, we show that for structural diversity in the poetry and code domains, CoS significantly improves diversity compared to several baselines.
Teaching Code LLMs to Reason with Intermediate Formal Specifications
FPT Software AI Center（FPT软件人工智能中心）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；Hanoi Univ. of Science and Tech.（河内科学技术大学）；Texas, USA（得克萨斯州，美国）；Hanoi, Vietnam（河内，越南）
AutoCAP: Towards Automatic Cross-lingual Alignment Planning for Zero-shot Chain-of-Thought
School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；Research Center for SCIR, Harbin Institute of Technology（哈尔滨工业大学SCIR研究中心）
Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs
University of Trento（特伦托大学）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核子研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（斯德哥尔摩大学）
BTC-SAM: Leveraging LLMs for Generation of Bias Test Cases for Sentiment Analysis Models
Insight SFI Research Center on Data Analytics, Dublin City University（数据分析洞察SFI研究中心，都柏林城市大学）；Laboratoire LITAN, École supérieure en Sciences et Technologies de l’Informatique et du Numérique（LITAN实验室，信息与数字技术高等学院）；School of Computing, Dublin City University（计算学院，都柏林城市大学）；Plus Que Pro, Strasbourg, France（斯特拉斯堡法国Plus Que Pro公司）；ServiceNow, Zurich, Switzerland（瑞士苏黎世ServiceNow公司）；Department of Information Engineering and Computer Science, University of Trento（信息工程与计算机科学系，特伦托大学）；Télécom SudParis, SAMOVAR, Institut Polytechnique de Paris（巴黎理工学院SAMOVAR，Telecom SudParis）
