ScatterShot：文本转换的交互式上下文示例管理
ScatterShot: Interactive In-context Example Curation for Text Transformation
摘要：
LLM（如GPT-3）的上下文学习功能允许注释者使用少量示例来定制LLM以满足其特定任务。然而，用户在创建示例时往往只包括最明显的模式，导致未指定的上下文功能在看不见的情况下不足。此外，即使对于已知的模式，也很难知道何时已经包括了“足够”的示例。在这项工作中，我们提出了ScatterShot，一个互动系统，用于建立高质量的演示集上下文学习。ScatterShot迭代地将未标记的数据切片为特定于任务的模式，以主动学习的方式从未充分探索或尚未饱和的切片中采样信息输入，并在LLM和当前示例集的帮助下帮助用户更有效地标记。在对两种文本扰动场景的模拟研究中，ScatterShot采样将产生的Few-Shot函数比随机采样提高了4-5个百分点，并且随着添加更多示例，方差更小。在用户研究中，ScatterShot极大地帮助用户覆盖输入空间中的不同模式，并更有效地标记上下文示例，从而实现更好的上下文学习和更少的用户工作。
英文摘要：
The in-context learning capabilities of LLMs like GPT-3 allow annotators to customize an LLM to their specific tasks with a small number of examples. However, users tend to include only the most obvious patterns when crafting examples, resulting in underspecified in-context functions that fall short on unseen cases. Further, it is hard to know when "enough" examples have been included even for known patterns. In this work, we present ScatterShot, an interactive system for building high-quality demonstration sets for in-context learning. ScatterShot iteratively slices unlabeled data into task-specific patterns, samples informative inputs from underexplored or not-yet-saturated slices in an active learning manner, and helps users label more efficiently with the help of an LLM and the current example set. In simulation studies on two text perturbation scenarios, ScatterShot sampling improves the resulting few-shot functions by 4-5 percentage points over random sampling, with less variance as more examples are added. In a user study, ScatterShot greatly helps users in covering different patterns in the input space and labeling in-context examples more efficiently, resulting in better in-context learning and less user effort.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
