Scalpel3:用于恢复碎片化文件的高性能数据雕刻架构
Scalpel3: A High-Performance Data Carving Architecture for Recovery of Fragmented Files
AI总结:
Scalpel3是首个通用开源大规模并行文件雕刻框架,支持连续与碎片化文件恢复,集成ONNX Runtime等实用功能,可降低文件雕刻研究向实用工具转化的门槛。
AI中文摘要:
文件雕刻是一种不依赖文件系统元数据从原始存储介质中恢复文件的技术,是数字取证、数据恢复和数字探索领域的关键能力。Foremost、Scalpel v1/2和PhotoRec等传统工具可有效处理连续文件,部分工具在简化假设下支持特定格式的碎片化恢复,但尚无公开工具提供通用、高性能框架,用于大规模开发和部署针对多种文件类型的碎片化恢复策略。本文提出Scalpel3,这是首个通用、开源、大规模并行的文件雕刻框架,支持大规模连续和碎片化文件恢复。其主要贡献是可扩展的高性能架构,使研究人员无需自行实现多线程后端或I/O基础设施,即可开发、测试和部署新的雕刻策略;新增文件类型支持通常仅需开发单线程文件验证逻辑,必要时还可开发块验证或自定义重组逻辑,Scalpel3后端会自动处理并行化、同步和快速I/O。该架构还集成了ONNX Runtime,必要时可让学习型分类器参与块验证和重组。除原始性能外,Scalpel3还引入了实际调查所需的实用功能:长时间运行任务的持久检查点、允许操作员监控进度并交互式重定向工作的人机交互控制,以及用于缩小搜索空间的块级重复数据删除。我们将该框架公开,旨在降低实验门槛,推动文件雕刻研究转化为从业者可用的工具。
英文摘要:
File carving recovers files from raw storage without filesystem metadata, a key capability in digital forensics, data recovery, and digital exploration. Existing tools recover contiguous files effectively, but, to our knowledge, no publicly available, format-agnostic, high-performance framework exists in which researchers can develop and deploy new fragmented recovery strategies. Scalpel3 fills this gap with a massively threaded architecture for contiguous and fragmented recovery. Researchers need only write single-threaded validation and reassembly code for a new file type; Scalpel3 supplies worker scheduling, synchronization, checkpointing, and I/O. This separation allows new recovery methods to be added without modifying the backend infrastructure. The architecture also integrates the ONNX Runtime, allowing learned models to be used within validators and recovery strategies. Operational features include interactive human-in-the-loop control, block deduplication, persistent restart checkpoints, incremental output, and a FUSE filesystem for hybrid workflows. We evaluate Scalpel3 on a mixed corpus of more than 80,000 files under contiguous recovery and three controlled fragmentation scenarios: gaps, out-of-order block placement, and both together. Results show fast and accurate contiguous recovery and demonstrate that Scalpel3's massively threaded architecture makes validated fragmented results available substantially earlier than single-threaded execution. Furthermore, strategies tailored to individual file types maintain high overall accuracy across increasingly difficult layouts. Together, these results demonstrate that Scalpel3 provides a practical foundation for developing and deploying fragmented recovery strategies at scale.