arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.15109cs.CRcs.LGcs.PF

Memory-Efficient Deep Learning Inference in Trusted Execution Environments

Worcester Polytechnic Institute(伍斯特理工学院)

Jean-Baptiste Truong, William Gallagher, Tian Guo, Robert J. Walls

英文摘要:

This study identifies and proposes techniques to alleviate two key bottlenecks to executing deep neural networks in trusted execution environments (TEEs): page thrashing during the execution of convolutional layers and the decryption of large weight matrices in fully-connected layers. For the former, we propose a novel partitioning scheme, y-plane partitioning, designed to (i) provide consistent execution time when the layer output is large compared to the TEE secure memory; and (ii) significantly reduce the memory footprint of convolutional layers. For the latter, we leverage quantization and compression. In our evaluation, the proposed optimizations incurred latency overheads ranging from 1.09X to 2X baseline for a wide range of TEE sizes; in contrast, an unmodified implementation incurred latencies of up to 26X when running inside of the TEE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑