Integrated ARM ***-Mali pipeline for high-throughput CNN inference

作     者：Aghapour, Ehsan Pathania, Anuj Ananthanarayanan, Gayathri 

作者机构：The Informatics Departments University of Amsterdam Netherlands The Department of Computer Science and Engineering Indian Institute of Technology Karnataka Dharwad India 

出 版 物：《TechRxiv》 (TechRxiv)

年 卷 期：2021年

核心收录：

主　　题：System on chip 

摘      要：Heterogeneous Multi-Processor System on Chips (HMPSoCs) combine several types of processors on a single chip. State-of-the-art embedded devices are becoming ever more powerful thanks to advancements in the computation power of HMPSoCs that enable them. Consequently, these devices increasingly perform the task of Machine Learning (ML) inferencing at the edge. State-of-the-art HMPSoCs can perform on-chip embedded inference on its CPU and GPU. Multi-component pipelining is the method of choice to provide high-throughput Convolutions Neural Network (CNN) inference on embedded platforms. In this work, we provide details for the first CPU-GPU pipeline design for CNN inference called Pipe-All. Pipe-All uses the ARM-CL library to integrate an ARM *** CPU with an ARM Mali GPU. Pipe-All is the first three-stage CNN inference pipeline design with ARM’s big CPU cluster, Little CPU cluster, and Mali GPU as its stages. Pipe-All provides on average 75.88% improvement in inference throughput (over peak single-component inference) on Amlogic A311D HMPSoC in Khadas Vim 3 embedded platform. We also provide an open-source implementation for Pipe-All under MIT License. © 2021, CC BY.