AutoSynth: Automated Workflow Optimization for High-Quality Synthetic Dataset Generation via Monte Carlo Tree Search
Shanghai Innavation Institute（上海创新研究院）；Shanghai Institute of AI for Education（上海人工智能教育研究院）；Department of Education Information Technology（教育信息技术部）；School of Computer Science and Technology（计算机科学与技术学院）
针对特定任务的大型语言模型（LLMs）监督微调（SFT）需要高质量数据集，但人工整理成本过高。合成数据生成具有可扩展性，但其有效性依赖于复杂的多阶段工作流，需整合提示工程和模型编排。现有自动化工作流方法存在冷启动问题：它们需要带标签的数据集进行奖励建模，这在无客观真值的主观、开放式任务中尤其棘手。我们提出AutoSynth框架，通过将问题重构为蒙特卡洛树搜索（由新型无数据集混合奖励引导），在无需参考数据集的情况下实现工作流发现与优化。该奖励通过两个LLM-as-judge组件实现元学习：一个利用动态生成的任务特定指标评估样本质量，另一个评估工作流代码和提示质量。在主观教育任务上的实验显示，尽管专家设计的工作流获得更高的人类偏好率（96-99%胜率 vs. AutoSynth的40-51%），但基于AutoSynth生成数据训练的模型显著优于基线（40-51% vs. 2-5%），并在某些指标上匹配或超过专家工作流，表明发现了超出人类直觉的质量维度。这些结果是在将人类工作量从5-7小时减少到仅30分钟（减少>90%）的情况下实现的。AutoSynth解决了数据中心AI中的冷启动问题，为主观LLM任务提供了可扩展、经济高效的方法。代码：https://github.com/bisz9918-maker/AutoSynth。
英文摘要：
Supervised fine-tuning (SFT) of large language models (LLMs) for specialized tasks requires high-quality datasets, but manual curation is prohibitively expensive. Synthetic data generation offers scalability, but its effectiveness relies on complex, multi-stage workflows, integrating prompt engineering and model orchestration. Existing automated workflow methods face a cold start problem: they require labeled datasets for reward modeling, which is especially problematic for subjective, open-ended tasks with no objective ground truth. We introduce AutoSynth, a framework that automates workflow discovery and optimization without reference datasets by reframing the problem as a Monte Carlo Tree Search guided by a novel dataset-free hybrid reward. This reward enables meta-learning through two LLM-as-judge components: one evaluates sample quality using dynamically generated task-specific metrics, and another assesses workflow code and prompt quality. Experiments on subjective educational tasks show that while expert-designed workflows achieve higher human preference rates (96-99% win rates vs. AutoSynth's 40-51%), models trained on AutoSynth-generated data dramatically outperform baselines (40-51% vs. 2-5%) and match or surpass expert workflows on certain metrics, suggesting discovery of quality dimensions beyond human intuition. These results are achieved while reducing human effort from 5-7 hours to just 30 minutes (>90% reduction). AutoSynth tackles the cold start issue in data-centric AI, offering a scalable, cost-effective method for subjective LLM tasks. Code: https://github.com/bisz9918-maker/AutoSynth.
Novelty-Driven Target-Space Discovery in Automated Electron and Scanning Probe Microscopy
Department of Materials Science and Engineering, University of Tennessee, Knoxville（田纳西大学材料科学与工程系）；Department of Chemistry, University of Washington, Seattle（华盛顿大学化学系）；Pacific Northwest National Laboratory, Richland（太平洋西北国家实验室）
Towards an automated workflow in materials science for combining multi-modal simulative and experimental information using data mining and large language models
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）；Karlsruhe University of Applied Sciences (HKA)（卡尔斯鲁厄应用技术大学）
RefGrader: Automated Grading of Mathematical Competition Proofs using Agentic Workflows
Pennsylvania State University（宾夕法尼亚州立大学）；City University of New York（纽约市立大学）；New York University（纽约大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）；Autodesk（欧特克公司）；Carnegie Mellon University（卡内基梅隆大学）
All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streams
Chair of Process and Data Science, RWTH Aachen University（过程与数据科学教授席位，亚琛工业大学）；Artificial Intelligence & Machine Learning Lab, Technical University of Darmstadt（人工智能与机器学习实验室，达姆施塔特技术大学）
Ensemble learning and iterative training (ELIT) machine learning: applications towards uncertainty quantification and automated experiment in atom-resolved microscopy
Integration of Scanning Probe Microscope with High-Performance Computing: fixed-policy and reward-driven workflows implementation
University of Tennessee（田纳西大学）；Oxford Instruments Asylum Research（牛津仪器庇护研究公司）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；National Cheng Kung University（国立成功大学）；the Pennsylvania State University（宾夕法尼亚州立大学）
Self-Optimizing Machine Learning Potential Assisted Automated Workflow for Highly Efficient Complex Systems Material Design
Key Laboratory of Material Simulation Methods \& Software of Ministry of Education, College of Physics, Jilin University, Changchun 130012, China；Research Center for Crystal Materials, State Key Laboratory of Functional Materials；Devices for Special Environmental Conditions, Xinjiang Key Laboratory of Functional Crystal Materials, Xinjiang Technical Institute of Physics；Chemistry, Chinese Academy of Sciences, 40-1 South Beijing Road, Urumqi 830011, China；State Key Laboratory of Superhard Materials, College of Physics, Jilin University, Changchun 130012, China；Department of Chemistry, University College London, Gordon Street, London, WC1H 0AJ, UK；International Center of Future Science, Jilin University, Changchun 130012, China；Key Laboratory of Physics；Technology for Advanced Batteries of Ministry of Education, College of Physics, Jilin University, Changchun, 130012, China
DeepWisdom（深度智慧）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Renmin University of China（中国人民大学）；Nanjing University（南京大学）；Fudan University（复旦大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Mila（米拉研究所）；The Hong Kong University of Science and Technology（香港科技大学）
