CLASS-IT：面向BabyLMs的对话与讲座对齐的小规模指令微调
CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs
AI总结：
该研究针对小规模语言模型，对比不同指令微调数据集与应用策略的效果，发现微调场景下有稳定提升但难迁移至零样本任务，为低资源LM的泛化优化提供了方向参考。
摘要：
本研究探讨小规模语言模型（LM）能否从指令微调中获益。我们对比了对话类与问答类指令微调数据集，采用合并或顺序课程两种应用方式，实验对象为参数量100M和140M的仅解码器模型。评估涵盖微调场景（SuperGLUE）与零样本场景（BLiMP、EWok、WUGs、实体追踪及心理语言学相关性）。结果显示，指令微调在微调场景中带来小幅但稳定的提升，其中顺序课程的表现优于合并数据；但这类提升无法稳定迁移到零样本任务，表明以交互为核心的适配与广泛语言泛化之间存在权衡。这些结果凸显了将类人学习策略适配于低资源LM的潜力与局限，也为在生态训练限制下提升泛化能力的混合式、基于课程的方法指明了方向。
英文摘要：
This work investigates whether small-scale LMs can benefit from instruction tuning. We compare conversational and question-answering instruction tuning datasets, applied either in a merged or sequential curriculum, using decoder-only models with 100M and 140M parameters. Evaluation spans both fine-tuning (SuperGLUE) and zero-shot (BLiMP, EWoK, WUGs, entity tracking, and psycholinguistic correlation) settings. Results show that instruction tuning yields small but consistent gains in fine-tuning scenarios, with sequential curricula outperforming merged data; however, improvements do not consistently transfer to zero-shot tasks, suggesting a trade-off between interaction-focused adaptation and broad linguistic generalization. These results highlight both the potential and the constraints of adapting human-inspired learning strategies to low-resource LMs, and point toward hybrid, curriculum-based approaches for enhancing generalization under ecological training limits.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
