Graph representation learning, a critical step in graph-centric tasks, has seen significant advancements. Earlier techniques often operate in an end-to-end setting, which heavily rely on the availability of ample labeled data. This constraint has spurred the emergence of few-shot learning on graphs, where only a few labels are available for each task. Given the extensive literature in this field, this survey endeavors to synthesize recent developments, provide comparative insights, and identify future directions. We systematically categorize existing studies based on two major taxonomies: (1) Problem taxonomy, which explores different types of data scarcity problems and their applications, and (2) Technique taxonomy, which details key strategies for addressing these data-scarce few-shot problems. The techniques can be broadly categorized into meta-learning, pre-training, and hybrid approaches, with a finer-grained classification in each category to aid readers in their method selection process. Within each category, we analyze the relationships among these methods and compare their strengths and limitations. Finally, we outline prospective directions for few-shot learning on graphs to catalyze continued innovation in this field. The website for this survey can be accessed by \url{https://github.com/smufang/fewshotgraph}.
Bidirectional Language Models Are Also Few-shot Learners
University of Pennsylvania(宾夕法尼亚大学);Microsoft(微软);Google Research(谷歌研究院);UNC Chapel Hill(北卡罗来纳大学教堂山分校)
大语言模型,例如GPT-3(Brown等人,2020)可以执行任意任务,而无需在仅使用几个标记示例进行提示后进行微调。一个任意的任务可以被重新表述为一个自然语言提示,一个语言模型可以被要求生成完成,间接地执行任务的范式称为基于语义的学习。到目前为止,基于突现的学习能力主要是针对单向语言模型来证明的。然而,在去噪目标上预训练的双向语言模型(例如掩蔽语言建模)为迁移学习产生更强的学习表示。这激发了促进双向模式的可能性,但它们的预培训目标使它们在很大程度上与现有的激励范式不相容。我们提出了SAP(顺序自回归拟合),一种技术,使双向模型的提示。利用机器翻译任务作为案例研究,我们提出了双向mT5模型(Xue et al.,2021)与SAP进行比较,并证明其Few-Shot和zero-shot转换优于单向模型(如GPT-3和XGLM)的Few-Shot转换(Lin et al.,2021),尽管mT5的参数减少了约50%。我们进一步表明SAP在问答和摘要方面是有效的。这是第一次,我们的研究结果表明,基于语义的学习是一个更广泛的一类语言模型,而不仅仅是单向模型的新兴属性。
Evaluating Prompts Across Multiple Choice Tasks In a Zero-Shot Setting
大语言模型已经表明,通过自然语言提示可以实现令人印象深刻的zero-shot性能(Radford et al.,2019; Brown等人,2020; Sanh等人,2021年)。然而,创建一个有效的提示需要大量的尝试和错误。这就向\textit{prompts}提出了一个问题:提示的质量如何影响它的表现?为此,我们从各种任务中收集并标准化提示,以用于它们不是为之设计的任务。然后,我们在固定的多项选择数据集上评估这些提示,以定量分析提示的某些属性如何影响性能。我们发现,包括选择和使用提示不使用在预培训提供了显着的改善。所有的实验和代码都可以在https://github.com/gabeorlanski/zero-shot-cross-task上找到。
Prompting Decision Transformer for Few-Shot Policy Generalization
Carnegie Mellon University(卡内基梅隆大学);University of Montreal(蒙特利尔大学);Mila(米拉);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室);Massachusetts Institute of Technology(麻省理工学院);UMass Amherst(马萨诸塞大学阿默斯特分校)
Meta-Prompting for Automating Zero-shot Visual Recognition with LLMs
ICG, TU Graz(格拉茨工业大学ICG研究所);CDL-EML(CDL-EML中心);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);JKU(约翰内斯开普勒大学);IBM Research(IBM研究院);Weizmann Institute of Science(魏茨曼科学研究所);University of Bonn(波恩大学)
Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance
Indiana University Bloomington(印第安纳大学伯明顿分校);Center for Complex Networks and Systems(复杂网络与系统中心)
PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning
School of Information Science & Technology, Pennsylvania State University(宾夕法尼亚州立大学信息科学与技术学院);Department of Computer Science, Stony Brook University(石溪大学计算机科学系)