arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.02609cs.CR
收藏

RLTrace:为操作系统模糊测试合成高质量系统调用轨迹

RLTrace: Synthesizing High-Quality System Call Traces for OS Fuzz Testing

Wei Chen, Huaijin Wang, Weixi Gu, Shuai Wang

AI总结:

本研究针对操作系统模糊测试中现有系统调用轨迹覆盖场景有限的问题,提出基于深度强化学习的RLTrace方法,能合成更全面的系统调用轨迹,提升Linux内核测试的代码覆盖率,并发现了一个未公开的内核漏洞。

摘要:

保障操作系统(OS)内核安全是当前网络安全领域的核心挑战之一。操作系统内核的前沿测试技术是软件模糊测试。模糊测试通过对程序输入进行多轮随机变异,旨在触发由可被输入利用的潜在漏洞导致的程序崩溃与挂起。为实现高操作系统代码覆盖率,主流的操作系统模糊测试工具通常会构建系统调用轨迹作为输入种子进行变异,并以此与操作系统内核交互。因此,所使用的系统调用轨迹的质量与多样性成为决定操作系统模糊测试效果的关键因素。但截至目前,这些系统调用轨迹要么通过手工编码规则生成,要么通过分析操作系统实用程序的系统调用日志生成。我们的观察表明,这类系统调用轨迹仅能涵盖操作系统系统调用的常见使用场景,很可能遗漏隐藏的漏洞。\n本研究提出了一种基于深度强化学习的解决方案RLTrace,用于合成多样且全面的系统调用轨迹,作为模糊测试操作系统内核的种子。在模型训练过程中,深度学习模型与操作系统内核交互,并围绕我们的学习目标——最大化内核代码覆盖率——推导出最优的系统调用轨迹。我们的评估显示,RLTrace生成的系统调用轨迹更全面,涵盖了系统调用的边界用例与微妙依赖关系,性能优于其他种子生成工具。将RLTrace合成的系统调用轨迹输入主流操作系统模糊测试工具SYZKALLER后,我们证实SYZKALLER在测试Linux内核时能实现更高的代码覆盖率。此外,RLTrace还在Linux内核(5.5-rc6版本)中发现了一个漏洞,据我们撰写本文时所知,该漏洞此前未被公开披露。

英文摘要:

Securing operating system (OS) kernel is one central challenge in today's cyber security landscape. The cutting-edge testing technique of OS kernel is software fuzz testing. By mutating the program inputs with random variations for iterations, fuzz testing aims to trigger program crashes and hangs caused by potential bugs that can be abused by the inputs. To achieve high OS code coverage, the de facto OS fuzzer typically composes system call traces as the input seed to mutate and to interact with OS kernels. Hence, quality and diversity of the employed system call traces become the prominent factor to decide the effectiveness of OS fuzzing. However, these system call traces to date are generated with hand-coded rules, or by analyzing system call logs of OS utility programs. Our observation shows that such system call traces can only subsume common usage scenarios of OS system calls, and likely omit hidden bugs. In this research, we propose a deep reinforcement learning-based solution, called RLTrace, to synthesize diverse and comprehensive system call traces as the seed to fuzz OS kernels. During model training, the deep learning model interacts with OS kernels and infers optimal system call traces w.r.t. our learning goal -- maximizing kernel code coverage. Our evaluation shows that RLTrace outperforms other seed generators by producing more comprehensive system call traces, subsuming system call corner usage cases and subtle dependencies. By feeding the de facto OS fuzzer, SYZKALLER, with system call traces synthesized by RLTrace, we show that SYZKALLER can achieve higher code coverage for testing Linux kernels. Furthermore, RLTrace found one vulnerability in the Linux kernel (version 5.5-rc6), which is publicly unknown to the best of our knowledge by the time of writing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑