面向大语言模型的基于关键词的采样（KEYS）
KEYword based Sampling (KEYS) for Large Language Models
AI总结：
针对问答生成中采样与人类关键词驱动表达关注不足的问题，提出KEYS方法，利用知识蒸馏提取关键词并引导解码采样，生成更类人且事实正确的答案，性能优于常用解码方法。
摘要：
问答（Q/A）可以被表述为一种生成任务（Mitra，2017），即在给定问题和段落（如果有知识可用）的情况下生成答案。问答任务的最新进展大量集中在语言模型的进步上，而对采样等其他领域关注较少（Krishna 等，2021）、（Nakano 等，2021）。关键词在人类语言生成中发挥着非常重要的作用。（人类会构想出关键词，并使用语法将这些关键词和内容连接起来。）在研究界，对于人类如何针对问题生成答案，以及如何将这种行为融入语言模型，关注非常少。本文希望结合探索这两个领域，即如何利用采样生成接近人类行为且事实正确的答案。因此，我们认为问答任务应采用的解码算法类型也应取决于关键词。这些关键词可以从问题、段落或互联网结果中获得。我们使用知识蒸馏技术提取关键词，并在普通解码算法之上利用这些提取出的关键词进行采样，以在组织答案时生成类人答案。本文表明，我们的解码方法在问答任务上优于最常用的解码方法。
英文摘要：
Question answering (Q/A) can be formulated as a generative task (Mitra, 2017) where the task is to generate an answer given the question and the passage (knowledge, if available). Recent advances in QA task is focused a lot on language model advancements and less on other areas such as sampling(Krishna et al., 2021), (Nakano et al., 2021). Keywords play very important role for humans in language generation. (Humans formulate keywords and use grammar to connect those keywords and work). In the research community, very little focus is on how humans generate answers to a question and how this behavior can be incorporated in a language model. In this paper, we want to explore these two areas combined, i.e., how sampling can be to used generate answers which are close to human-like behavior and factually correct. Hence, the type of decoding algorithm we think should be used for Q/A tasks should also depend on the keywords. These keywords can be obtained from the question, passage or internet results. We use knowledge distillation techniques to extract keywords and sample using these extracted keywords on top of vanilla decoding algorithms when formulating the answer to generate a human-like answer. In this paper, we show that our decoding method outperforms most commonly used decoding methods for Q/A task
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
